{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Improving Computer Vision Accuracy using Convolutions\n",
    "\n",
    "In the previous lessons you saw how to do fashion recognition using a Deep Neural Network (DNN) containing three layers -- the input layer (in the shape of the data), the output layer (in the shape of the desired output) and a hidden layer. You experimented with the impact of different sized of hidden layer, number of training epochs etc on the final accuracy.\n",
    "\n",
    "For convenience, here's the entire code again. Run it and take a note of the test accuracy that is printed out at the end. \n",
    "\n",
    "# 利用卷积提高计算机视觉精度\n",
    "\n",
    "在前几节课中，你看到了如何使用包含三层的深度神经网络（DNN）进行时Fashion MNIST图像识别--输入层（根据输入数据的形状）、输出层（根据类别数量）和一个隐藏层。你实验了不同大小的隐藏层、训练次数等对最终精度的影响。\n",
    "\n",
    "为了方便起见，这里又提供了完整的代码。运行它，记下最后打印出来的测试精度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "height": 207
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 19572,
     "status": "ok",
     "timestamp": 1550247198665,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "xcsRtq9OLorS",
    "outputId": "027ddd16-b2d9-41a0-85aa-9da6275085e9"
   },
   "outputs": [
    {
     "ename": "",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31mFailed to interrupt the Kernel. \n",
      "debug session not found. \n",
      "View Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details."
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "mnist = tf.keras.datasets.fashion_mnist\n",
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
    "training_images=training_images / 255.0\n",
    "test_images=test_images / 255.0\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(128, activation=tf.nn.relu),\n",
    "  tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "])\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "\n",
    "test_loss = model.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your accuracy is probably about 89% on training and 87% on validation...not bad...But how do you make that even better? One way is to use something called Convolutions. I'm not going to details on Convolutions here, but the ultimate concept is that they narrow down the content of the image to focus on specific, distinct, details. \n",
    "\n",
    "If you've ever done image processing using a filter (like this: https://en.wikipedia.org/wiki/Kernel_(image_processing)) then convolutions will look very familiar.\n",
    "\n",
    "In short, you take an array (usually 3x3 or 5x5) and pass it over the image. By changing the underlying pixels based on the formula within that matrix, you can do things like edge detection. So, for example, if you look at the above link, you'll see a 3x3 that is defined for edge detection where the middle cell is 8, and all of its neighbors are -1. In this case, for each pixel, you would multiply its value by 8, then subtract the value of each neighbor. Do this for every pixel, and you'll end up with a new image that has the edges enhanced.\n",
    "\n",
    "This is perfect for computer vision, because often it's features that can get highlighted like this that distinguish one item for another, and the amount of information needed is then much less...because you'll just train on the highlighted features.\n",
    "\n",
    "That's the concept of Convolutional Neural Networks. Add some layers to do convolution before you have the dense layers, and then the information going to the dense layers is more focussed, and possibly more accurate.\n",
    "\n",
    "Run the below code -- this is the same neural network as earlier, but this time with Convolutional layers added first. It will take longer, but look at the impact on the accuracy:\n",
    "\n",
    "你的训练准确率大概是89%，测试准确率大概是87%......还不错......但是如何让它变得更好呢？一种方法是使用一种叫做Convolutions的方法。这里不打算纠结于卷积的细节，而想抓住它的核心思路，即通过卷积操作缩小了图像的内容，将模型注意力集中在图像特定的、明显的特征上。\n",
    "\n",
    "如果你曾经使用滤镜进行过图像处理（比如：https://en.wikipedia.org/wiki/Kernel_(image_processing)），那么convolutions看起来会非常熟悉。\n",
    "\n",
    "简而言之，如果取一个二维数组（通常是3x3或5x5）并将其应用到图像上。通过根据该矩阵内的公式改变底层像素，就可以进行图像边缘检测等工作。例如上面的链接，会看到一个3x3的矩阵，它是为边缘检测而定义的，其中间的单元格是8，而所有相邻单元格都是-1。在这种情况下，对于每个像素，把它的值乘以8，然后减去它周边像素的值（因为每个都乘了-1）。扫描整个图像，对每个像素都这样做，最终会得到一张边缘被增强的新图像。\n",
    "\n",
    "这种计算对于计算机视觉来说是非常理想的，因为通常情况下，能够像这样被突出显示的特征才是区分一个物品和另一个物品的关键。卷积使得所需要的信息量会少很多......因为只需要对突出显示的特征进行训练。\n",
    "\n",
    "这就是卷积神经网络的概念。在全连接层之前，增加一些层来做卷积，那么输入全连接层的信息就会更加集中，也可能更加准确。\n",
    "\n",
    "运行下面的代码--这和前面的神经网络是一样的，但这次先加了卷积层。这会花费较长的时间，但看看对精度的影响。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "height": 207
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 19572,
     "status": "ok",
     "timestamp": 1550247198665,
     "user": {
      "displayName": "Laurence Moroney",
      "photoUrl": "https://lh3.googleusercontent.com/-RcxktLY-TBk/AAAAAAAAAAI/AAAAAAAAABY/b4V4dTIqmPI/s64/photo.jpg",
      "userId": "06401446828348966425"
     },
     "user_tz": 480
    },
    "id": "xcsRtq9OLorS",
    "outputId": "027ddd16-b2d9-41a0-85aa-9da6275085e9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.4969 - accuracy: 0.8265\n",
      "Epoch 2/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.3727 - accuracy: 0.8669\n",
      "Epoch 3/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.3372 - accuracy: 0.8781\n",
      "Epoch 4/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.3131 - accuracy: 0.8859\n",
      "Epoch 5/5\n",
      "1875/1875 [==============================] - 2s 1ms/step - loss: 0.2955 - accuracy: 0.8921\n",
      "313/313 [==============================] - 0s 897us/step - loss: 0.3591 - accuracy: 0.8744\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "mnist = tf.keras.datasets.fashion_mnist\n",
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
    "training_images=training_images / 255.0\n",
    "test_images=test_images / 255.0\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(128, activation=tf.nn.relu),\n",
    "  tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "])\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "\n",
    "test_loss = model.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.1.0\n",
      "Train on 60000 samples\n",
      "Epoch 1/10\n",
      "60000/60000 [==============================] - 15s 250us/sample - loss: 0.1454 - accuracy: 0.9560\n",
      "Epoch 2/10\n",
      "60000/60000 [==============================] - 14s 227us/sample - loss: 0.0510 - accuracy: 0.9846\n",
      "Epoch 3/10\n",
      "60000/60000 [==============================] - 14s 226us/sample - loss: 0.0316 - accuracy: 0.9899\n",
      "Epoch 4/10\n",
      "60000/60000 [==============================] - 13s 224us/sample - loss: 0.0215 - accuracy: 0.9933\n",
      "Epoch 5/10\n",
      "60000/60000 [==============================] - 14s 238us/sample - loss: 0.0146 - accuracy: 0.9954\n",
      "Epoch 6/10\n",
      "60000/60000 [==============================] - 14s 240us/sample - loss: 0.0104 - accuracy: 0.9965\n",
      "Epoch 7/10\n",
      "60000/60000 [==============================] - 14s 228us/sample - loss: 0.0087 - accuracy: 0.9973\n",
      "Epoch 8/10\n",
      "60000/60000 [==============================] - 13s 225us/sample - loss: 0.0057 - accuracy: 0.9983\n",
      "Epoch 9/10\n",
      "60000/60000 [==============================] - 14s 232us/sample - loss: 0.0060 - accuracy: 0.9979\n",
      "Epoch 10/10\n",
      "60000/60000 [==============================] - 15s 243us/sample - loss: 0.0052 - accuracy: 0.9983\n",
      "10000/10000 [==============================] - 1s 132us/sample - loss: 0.0552 - accuracy: 0.9864\n",
      "0.9864\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "mnist = tf.keras.datasets.mnist\n",
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
    "training_images=training_images.reshape(60000, 28, 28, 1)\n",
    "training_images=training_images / 255.0\n",
    "test_images = test_images.reshape(10000, 28, 28, 1)\n",
    "test_images=test_images/255.0\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(28, 28, 1)),\n",
    "  tf.keras.layers.MaxPooling2D(2, 2),\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10, activation='softmax')\n",
    "])\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
    "model.fit(training_images, training_labels, epochs=10)\n",
    "test_loss, test_acc = model.evaluate(test_images, test_labels)\n",
    "print(test_acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1 3 1 3 4 7 2 7\n",
      " 1 2 1 1 7 4 2 3 5 1 2 4 4 6 3 5 5 6 0 4 1 9 5 7 8 9 3 7 4 6 4 3 0 7 0 2 9\n",
      " 1 7 3 2 9 7 7 6 2 7 8 4 7 3 6 1 3 6 9 3 1 4 1 7 6 9]\n"
     ]
    }
   ],
   "source": [
    "print(test_labels[:100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.1.0\n",
      "Model: \"sequential_9\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_14 (Conv2D)           (None, 26, 26, 64)        640       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_12 (MaxPooling (None, 13, 13, 64)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_15 (Conv2D)           (None, 11, 11, 64)        36928     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_13 (MaxPooling (None, 5, 5, 64)          0         \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 1600)              0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 128)               204928    \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 243,786\n",
      "Trainable params: 243,786\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 60000 samples\n",
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 26s 438us/sample - loss: 0.4373 - accuracy: 0.8406\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 25s 424us/sample - loss: 0.2910 - accuracy: 0.8939\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 25s 422us/sample - loss: 0.2491 - accuracy: 0.9076- loss: 0\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 25s 423us/sample - loss: 0.2159 - accuracy: 0.9197\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 25s 413us/sample - loss: 0.1904 - accuracy: 0.9288\n",
      "10000/10000 [==============================] - 2s 218us/sample - loss: 0.2571 - accuracy: 0.9094\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "print(tf.__version__)\n",
    "mnist = tf.keras.datasets.fashion_mnist\n",
    "(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
    "training_images=training_images.reshape(60000, 28, 28, 1)\n",
    "training_images=training_images / 255.0\n",
    "test_images = test_images.reshape(10000, 28, 28, 1)\n",
    "test_images=test_images/255.0\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1)),\n",
    "  tf.keras.layers.MaxPooling2D(2, 2),\n",
    "  tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "  tf.keras.layers.MaxPooling2D(2,2),\n",
    "  tf.keras.layers.Flatten(),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10, activation='softmax')\n",
    "])\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
    "model.summary()\n",
    "model.fit(training_images, training_labels, epochs=5)\n",
    "test_loss = model.evaluate(test_images, test_labels)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWcAAAD7CAYAAAC2a1UBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABVbUlEQVR4nO39eZRl11XgCf/2ucMbY47IOVOp1GBNtixP2NjYomSDKAyiqxuXVcu0q4v1QX003bCq+sPCvRas4vvodlG1WEUXFFXuwg0Uk1zYxgYMRggL2S4jSxaSNUspKVM5R2bM8YY7nf39cW9ERsSLzIx5PL+1It97+91377k739v33H32IKqKw+FwOLYWZrMH4HA4HI5OnHF2OByOLYgzzg6Hw7EFccbZ4XA4tiDOODscDscWxBlnh8Ph2IKsyjiLyL0i8pKIHBeRB9ZqUA6Hw7HbWbFxFhEP+A3gB4DbgPtF5La1GpjDXfwcjt2Mv4rPvgs4rqqvAYjIHwH3Ac9f6QMistszXi6p6tBSNpxz8fsQcBp4XES+pKqL6tfpdum6hfzCB/wa4AH/WVU/dY3td7V+VVXWa9+7Xbdc4bu7GuN8EDg15/Vp4Luu/TFvFYfc7mQnl7Hxsi9+TrdLY7kXvsvsVv1mG3CM3apbuNJ3d90XBEXkJ0TkCRF5Yr2PtcNY7OJ3cJPGstOYvfCpagzMXPgcji3DaozzGeDwnNeHCtk8VPXTqvoOVX3HKo7lWAR34VsxS7rwOf2uDLdWsjasxjg/DtwkIteLSAh8FPjS2gzLwRIufu7Ct744/S4fFyiwdqzYOKtqCvw08BXgBeCzqvrcWg3M4S5+68iS7vocK8K5jNaI1SwIoqpfBr68RmNxzEFVUxGZufh5wGfcxW/NmL3wkRvljwL/ZHOHtGNYYaCAYyGrMs6O9cVd/NYHd+HbfETkJ4Cf2OxxbGWccXbsStyFb91YcqAA8Glwcc5XwtXWcDgca4lbK1kj3MzZ4XCsGc5ltHY44+xwONYU5zJaG5xbw+FwOLYgzjg7HA7HFsS5NZgptrX6BWMjNcKgD2tTkvQSSrrqfTocjt3JLjfOguCBGFQTVmugK+E+jnlvoyVNTuh/I83G1maY24jvq3aGrv5V89Mr3t93Vf7HDtljrd9d8f4cju3CLjXOl42yZ2oAZLZBnm268n0aMYjzFDkcjjVgVxpn3+unFu4lMBX6OYBgOJU+TTM6sYK9CSIljJSIswYn5RnSLCKzrbUetsPh2EXsSuMc+l10eXuoaQ/77SCeGC55fTQ5saL9GSnhmQqZjWjE57CaFm4Sh8PhWBm7zDgLYBA8PHwiafOGOQuAT4meym1E2STt+BzL6f5gNUKtBbUoGWCLP4fD4VgZu8w4G0Q8fFMi0BINxhluPYtqysHad/EmbudscJ7XkjGsNpa4T0W1jWp7XUe+Vbm39pPzXr/G2Y5tvvP9H+iQ/atv3t4h+3r2ZIfssYZb/HPsTnaRcfbwTB1jQgRDJC1i28RqC9UMxWKAunbTV7mZ2E4TpVNktoVqilK4KTSfESvKxvRWczgcu5FdY5w9U+dw9d102W7OyWucaz2N1bjwDRtibTJNxDG/l/sH30lglCdGAl5pTzFtprnIG1hNyTRBNaOVjpKkl1iL+GiHw+FYyDWNs4h8BvgwMKyqdxSyfuBB4ChwAviIqm7poF5jQoayPfRKhQt4pNnIvPdTjYgkojvo4V17z1ANIxrp9bSzOmNJicTERKZFpgkZCalGhXF2OByOtWcpM+ffBn4dmOv8ewB4WFU/VTRwfAD4xNoPb/UYqRH4PZS8LsbMKE0t05pNDrns6ihJHRTOtBK+8PphAqO8Pq2cS5pEhUsj0BJ1egg0BB/a8UVUo2IRUDGmi5Lfh2KJk7F5760EETkBTJH7T1LXy87h2D1c0zir6qMicnSB+D7g7uL57wCPsEWNcxj00R9cD8CYniXThFYyCoBIQDXcQ2jqlKSOqOEVc5JvjL2E1ZSqP0DJ1PGlREkrlCgzZPupG5+EmFHvVTJrwLZQMsrBAAf820kk4gIvEqcToNEqk1v4XlXdElP0WumGDlmm8y88t3mHOrb52NdGO2Rd2hkH/g7vrR2yE9WOptic43iHbLTZuZjocGxnVprOtldVzxXPzwN7r7Th5rWX9xB8AlOhSjeBlEg1IrbT2MJYigSUvG6qpo9QQxRLSkScThCnE2SaIBgMBg8fX30yLC2bkZFiJMCIj0gJkQBfSnj4eBrgSQkjIa62lMPhWAmrXhBUVb1am5nNaUfj4XvdeKZCj3+QvdkeJmWa8+nzxOklVPMoC9/UOGpvpYsyEzSZNlPE2iyy+yyBqdCne/DUp6QhinLRXKIlDVo6gW8qeCacPWrZ6wHAYCh73RgxtBJLmq04zE6Bvyr09p8KXc7i+rA5HDuXlRrnCyKyX1XPich+YHgtB7VaBMEzFUKvRllrhEXSSWbnuxg8E1KnRN3zmc48YonJbALkCSUeAQEhvnqE+MSktKRBw46QaYIRAxiMBPn+CGb37UsJNZa2rOr69z5VPSMie4CHRORFVX105k3Xh83h2Lms1HJ8Cfg48Kni8YtrNqJVIBLimVo+Yw4PUZI6bRq8aF6gZSfI7ExiiYeIh9WUs2aYki1zyZxlMjlPOlMTQ3xSjWjKNCKGqeKTBkNZuskkj9pQtSS2iWLJSMhI8fDZr9djVDgRwugKQ+5U9UzxOCwiXwDeBTx69U85loJbbF0fROQwefDAXvIv/adV9dc2d1Tbk6WE0v0h+eLfoIicBn6R3Ch/VkR+HDgJfGQ9B7lUjFSoBIOEps6QHqZiK7wuzzHS/A55OnVuIEUCPFNB1XIuexHB0EyGixKfueEWCXLjbKawmpESIRjK1KnSTSppHlJHREIzD7HThNSklGyZg6aHqmcYTw8wyneWfS4iUgOMqk4Vz78P+KW109bVKQUHOmS3me/ukLV0fs3q5+wLHduEptohu6SdmYR17e2QpdJZE/vfHrmrQ/bPXljRguCWWWzdQaTAv1TVJ0WkC/i2iDykqs9v9sC2G0uJ1rj/Cm/ds8ZjWQUeACI+gkGxTJsJIm0TZ9Pkk6M8bE7Ex0iAiEHEkNoIxWJtsUiI4JkaRkoAJNomIyGxLQSDZwJEvCJEDqRICUdByWtrGAx1z9AdCpWkSr4ouOxswr3AF0QE8v+nP1DVv1ytphyO9aQIFDhXPJ8SkReAg4AzzstkB2QICsZUMRISeDVEPBLb5Gz8NNbG2CJkyzN1DlXfSd32MG0maOkk7WyC6fgsWoTCAYip0FM6ii8lEm3RysaIbYM4GQMxJEGLsteNLyUq0gNSGOjiApFoPsM+Uof9lZjXGv2spPWwqr4G3LlGSnJ0ctXFVnALrqulCMG9C3hskfecbq/BDjDOYMTHMyVEDKpZ7l7IJuYt/on41G0P3VrHqiUhwkhQJIqk8/YVSnXWOKcakdkIq01QQ2pbpKaEVywCmjmhclYzMskTViqe0uWnlCREEJfkvfW46mIruAXX1SAideBzwM+q6uTC951ur80OMM5KZhtYG5OIwYiPqp0Nl5vJEPRNhTEZpmGmaekEkZ0mttOzhYyM6cI3NTwTEmuTWJs00xGSdAqrEflEy6JqSW2ELyUySfKFQI1Ii9l37lKZZCwepOyFWKAcHiCzMXF6ZtO0tFyipNMn/FrwbIfML9w/M0wu8rkPhv+wQ/YcJztkTZnqkLXsRIfshcnrO2TLxS22rh8iEpAb5t9X1c9v9ni2KzvAOINqjBKDdlZR9r0uesJDWDIadoQpPV8UL7KFQVVACLwuasEQVrMiUcUSpxNYO99gWE2wxeJfprlxTm1UhOnlxrtlJhmNoOx5QEZXsI9ME0a2kXHeyWz2YutORvJFkt8CXlDVX93s8WxndoRxXgyRECFf+Es1wmpWGNbcgObdSiyCAB4lr4suM0SiEZFOk2qEZ8qoJggeUsQrGwnyBcTCMFsyjAR4poSqJbMxTTvGmXabdlZiysZUvG6sWEauPmTHxuEWW9eP9wI/BjwjIk8Vsk+q6pc3b0jbkx1pnAWfUrCH0NRINaKRXERkjm/Yplgtkk3ERySg3z/KDfYITY25YIaJTSv3O9smvilTkjqWjKn0PHE6hRGfrCiIVPLq+KZEK8nLiCbZKF83F/DSEkOlmzmYHcEgnNokfTjm4xZb1w9V/Tp5yyHHKtmRxpnC92wkAI1yQ6zMhtDNkM+cPQSDrz6BGDz1MORdtH0pIZ4hoEyFPPFkGo8Z37Mlw+AVx/KJjJ+7STQhzcZJM2iH+wnwMe776nA4lsEGG2dBJCwK3K/HAq2Qr0UYomSMWCYK90WESAnfr+QG24DaDDBYG6EacTZ9jil/hFQi2mm+uNzjH6CHITx8KrZCIgmjcjIvum/jfGHQlOhmDyXK1MI+pv0hFFvEPxvK0sVFsxEODQ9juq65Vej3dMgOBZ2TyOu0MwllcpFWXGfkxLzXR4LOBJE/a3ymQ/Y95R/rkD3NNztkb+bdHbKHxrZ06XCHY03YYONsEALAzgtfWyvyWXAe4pbZSRZeAEQMvimhNsOSoqSz7aaa0Sla0WkoZtYiJfqCw/RkvfgYAnwiLZJcikVBqwmqAXW66dYaderUJTd+Hj4oTDPJOBfW/FwdDsfOZoONs6IkhUFcSN4ZO38mxdYzhepnumYLYioYCfNynSY3lrZIIZ5xZSgZSdaYzfpTLEbCvKSnBEhRoznVmQiLvEegagaaFXHJCdPZJYY9H8Ej1JBYYqJoCjTFakyUTZFqxCX/LE3pJpGIuEh6keJcYppFlqLD4XAsnQ02zvaKhefziIhSnl5dlOGcSSQRPIypYUxId3iIqumjql10224MQiR5waEZUkmZCsZmU69VLSIGjyCvzFx0346lxbSpk2lEMx4m0zymVgE0ZaL1IhNyfHZ8Sla4ZDLUThPZJoLQis/Pvn85mG9uHeeFAX4Oh8NxdTberSFl0PSq7ZtmUqEXk4vkxe999QmL4WdqZ9eHLRZf/Tw5QsCoRyJtTLFPO6fGhWAw4mHVzFsonEFJoUhm0Vlja2ffhWzWkF/dg+4WAx0Ox/LYUONcl37uLP8Ir5mXudh+Easx1k6TuzsyVNuIymw9jHyWmrs3rG1gtcV4O2JSSngmJDAVALJZY5/jSUBJuvEIaGVjtNLRYn9FNmARtaFqi+PaOeVEL+OZHryioH7o1QFoxBfybTVdht98/bNT33bE8nc/35wnk4Fyx3ZP/+atHbLv/tq3O2TH0z/vkA3W3t4hG229PO/1Ix++vWObI+/p/Jr5n3h/h+xfH+vcLtHOC1s76zyvp5odIodjW7OhxrniGd7cVSWeOsa4d4rUGqxtwuwsemYmuvCTWsxiIc0uu0U6u9DliIRQuo6S1537hdOx2YW/BVsW0R3MpnvPfc8zFSpBPyVTp84AViyJ3yRK0nz+rGu/qOlwOBywwcbZN8pQyfJ+v4t3Zj9EaiFVyCxEVomtElllKkuISBgxI0wxQjMboRGdXXI3a9WMKJ3KU6xnU7QtM6VFfa+X0O+iy9/HTfY2quJT8TxCk8/SrEJg4Fgd9pSTvBK0Cs3M8HeXruc1/wIRbWJpYecYdSPe7EJgnkWY14FONQJcE1KHw7F0llJsf9HOBiLSDzwIHAVOAB9R1asGoIZGubG7wdsPn+C6O1/EBCmmlLsuoks9tMa7aEx0cerCPqaiMn8/cpiTjaO81mjzrfBh4nSKLJtYgjshI04vEKeGy0X281mySMBQ+RYO2WPcUurmH18/zN6uCQYHR+gezIdvM4Nfjql/7wTRW74Hf+wM3snj2NNt/vq/3MfXLuwnsdDOZDaWBMAT8CWPcs40N/LTCUylFqvwJzjj7HA4lsZSZs6LdjYA/inwsKp+SkQeAB4APnG1HeXGKiCKQ2ySH1qCFDGKeJagHFNO2vTWGgRexr5mjdiWSWyZc9GdNIMpWjpBavOZqC4SBaGaFZXiCn+12tntpOj3V6GbEB9PwKpgVdDib/6Abe66MB5UqpiuJgO1aQ5WeomtIbKCzpnEewJeUf0wUSFToRUYmpnBKtBZdM3hcDgWRVSXt1glIl8Efr34u3tOk9dHVPVNV/ts2evTY5UPss8OMBiGBAbKnuALDJUt/WFGPUg4VJsi9DMqQUzJT7BzjKbvZXheRpZ5pJk3z6DGmU+UBsSpx6V2lVbqE1lDOzO0MsOphkcjVS5EMZdkAg+fmpYJMFSNT9kzeCKERggMHK5aBkop+6tN3rT3LIGfMlN6Ns08sswnywxJ5pNkHqGXEvgpqkKS+WTWUAkjqpXcO37rnz/67YW96kTkM8CHgWFVvaOQLfuuJK+JOz/K5Qfr/7xju9++/687ZL0/sshVY2y8Q5Sd7oxoeeGv57eu+leP3dyxzZ81P9sh+61bfrBDdt8H/6ZDFjcqHbLuG093yMoPTHfodi1ZTL+7hwztmLmsHbtbtwDZot/dZfmcF3Q22Fu0pAE4T+72uCqRHeeFxhd4AYrVPG+2i8kN5fdxo+znulqVmp8yIE327B2hb+8lSr1TlI9cQqpATxVKZUhiaBbpxDafGWvDYqcCbLPExOsHaDeqxFGJVrvERKPOs5f2MBoHJKMhp7Imk1zkYus76Jy0ZMHH87oQ8ak2hyiZOm/Vt3Jf+xh7yi3uOnacwevOYjMPm/jYzBA3KyRxQFiK8UsxqkIahdjM0LVnlMqBSyAKnQEQAL9NfqH73TmyB1jmXYnD4dhZLNk4L+xsUJRbBEBV9UrdDK7ejsaiGmHVclFPoFjGJwdppt3U/G4OXBqkN0ypeBl95TaBySj5Kb7JsNaQWG+eWyHOfOLMI7IeI1GJdmbICvdCnAmjsSG2ENuMPtsPBsa9XtJsCqtt8miRjMy2EAnyKnNek+P+KR65cD3dQZlXpu5i6MXbsUBmDQq0M49UITBKILkTJbX5LPNNvWPcsO8sxijwtQ4NqOqjxUVvLveRN9UF+B3gEZxxdjh2FUsyzlfobHBBRPbPcWsML/bZq7ej0dlC+aPNZxnjOV4Rn29GQVE4yM8bsRbJJ/k+5leV6zxeMYsu4pd9r0JoaoRenQN6IzUtU5WQfV6NUhYwGu6llYZEyQhWG8WY2qi2ie00cQqvR6c5YSr5mMb8joSVueOYeU/VYiTgvWc/zAdH+vEEFjPOV2BJdyWuD5vDsXNZSrTGlTobfAn4OPCp4vGLqxvKnGy7RaqfrXivNiSRCrHfxUTYR6Q1rO0BW6JNPNvNZHG0+DdF7UpW84SzpYucbBzC73TXLomr3ZW4PmwOx87lmguCIvI+8infM1zOXf4kud/5s8AR4CT5otXoNfa1CY7/omiSeHimK6+xYUoY8bGaEidjl4sedSSprJ7AH6KndASAS41vLer4L9wafzZnQfAllrnY6hZVFl9UWSt2t37dguD6ssIFwWt0NrhntcNaf4rMQ81Is7yucrKBR0/Si1xKLy73Y2t8V+JwbCwi4gFPAGdU9cObPZ7tyApvth1rhYj8IfBN4E0iclpEfpzcKH9IRF4BPli8dji2Ez8DeWCWY2XszDZV2whVvf8Kb22DuxKHoxMROQT8IPDLwL/Y5OFsW9zM2bFjEZHPiMiwiDw7R9YvIg+JyCvFY99mjnGH8u+An+MqhcxF5CdE5AkReWLDRrXNcMbZsZP5beDeBbKZBJ+bgIeL1441QkRmsl0769DOQVU/rarvWM9F3O2OM86OHYuqPgosjCC6jzyxh+LxRzZyTLuA9wI/LCIngD8C/oGI/N7mDml74oyzY7ex5LID7tZ7+ajqz6vqIVU9CnwU+BtV/dgmD2tb4hYEHbuWqyX4FO+7JB/HpuFmzo7dxoUisYerlR1wrB5VfcTFOK+cjZ45X4KskT9uawZZ2Tlct9YDmcMlyE4Wz1c6vq3Ecs9hqbpdaYLPjH53gm6Xysy5ruf3FuZ/dxc7/maxUcdfVL/Lrue8WkTkie2+QrvVz2Grj28prMU5FAk+d5P/yC4Avwj8CcssO7DW49oubPa57vbjO5+zY8fiEnwc2xnnc3Y4HI4tyGYY509vwjHXmq1+Dlt9fEthq57DVh3XerDZ57qrj7/hPmeHw+FwXBvn1nA4HI4tiDPODofDsQXZUOMsIveKyEsicrzoKr3lEZHDIvJVEXleRJ4TkZ8p5Fuuutl21C9sn+px21W/12Kz9X8tvYpISUQeLN5/bJGGyKs59qK/7wXb3C0iEyLyVPH3C2t1/KuiqhvyR96H5lXgGBACTwO3bdTxVzHu/cDbiuddwMvAbcCvAA8U8geAf73J49yW+i3G/n7gbcCzc2ROv7tA/0vRK/BTwH8snn8UeHANj7/o73vBNneTt5Hb0P+XjZw5vws4rqqvqWpMXrHqvg08/opQ1XOq+mTxfIq8u8NBtl51s22pX9g21eO2rX6vxSbrfyl6nTuWPwbuKRpPr5qr/L43nVUZ52Xe5h0ETs15fZotooSlUtxO3UXe3HbJ1c02iG2v3wU4/W4uG6X/peh1dhtVTYEJYGCtB7Lg972Q94jI0yLyFyJy+1ofezFWbJyLBo6/AfwA+W3+/SJy21oNbKshInXgc8DPqurk3Pc0v/dZ85jEnerjXC7rpV/H0tgN+r/a7xt4ErhOVe8E/j15CYD1ZxW+mvcAX5nz+ueBn7/G9rrL/y6upS9uwfabfW6b/bdk3Rb6uhd4CThO4Vu9xvabfX6b/ffSevhVcXZBucJ3dzW1NRa7HfmuhRuJyE8AP3FZ4q3ikNudRStvXYlZXxyAiMz44p6/8kecbpfCnLu+D5F/bx8XkS+p6lV0C7tXvxksvXrfcnk8f9ituoUrfXfXfUFQXa+wlbLbfJwbyY5d3FtHPrUeOy18yI5FWI1xPgMcnvP6UCFzbBCujdKKWdKFz+n3Mrq8sqpurWQNWI1xfhy4SUSuF5GQPP7wS2szLAdLuPi5u5L1xel3+ey2QIH1ZMXGubgd+WngK+SxgZ9V1efWamAOd/FbR9xd3/rhXEZrxKqK7avql4Evr9FYHHNQ1VREZi5+HvAZd/FbM2YvfORG+aPAP9ncIe0YVhgo4FiI64SyhXEXv/XBXfg2H3Wdza+JM86OXYm78K0bzmW0RriSoQ6HYy1xayVrhJs5O1bFm6s/Ou/1M83/usZHWCw5IVvjYzjWCucyWjt2mXGe+aFb8qxJh8Ox1jiX0dqwa4yzSJmu0nUEpspkfJokvbiWey8encF3OBxrw64xzkZK9PmHqWoXsT+9xsYZcve9m5E7HI61YccbZ9/roxIMUvK66bNDlDQgMNV1OJIzzA6HY+3Y8ca5t3yMW+ydlNSj7uWne8p2r/FRdodRfkv1H3fIPrq3d97r37vp7o5tPvfirR2yRy/GHbJpog7Zt9t/3CFTdQuCjp3PjjXOgg/iU5Ee+rwQI0KqSpbXkEUkRAgI/B5EDJmNUE1RtSgWANVkwV7t5adqUVxBLYfDsT7sSOMsEtJdvpGaGeAWeyM39HiMRfC38XHG7Cl8KTFUfStH7M1830A33UHGG02fs01LokojS0iwTMo0DZme3a8VS0aCYplMzzEdnUZJCiOugCB4KIoL93I4HKthZxpnArq9fQzYvQyVfAZKGc3UYyR9jan2cXoqt7JHjnJD2M3dB84z1D3Bc+cPcHyqRjM1jMdlUoWL7RJTWffsfNmiJKRkWPChmQxjLSgZuTE2ID6CRdX5oB0Ox8rZUcbZSI3A76HkdVHRGkaF0+2IiSRgImvhSYlKeJj3e+/n7r2W/ZVJju47S6Xa4lYrDFZ6aKUBY+0y7czjTCtkLC7TTGEysWQKiQ2wQCm7nmq5i5SUNtOkRNQZoNf2UaXMdeUKVR8eb4zyTPvLqKbYjtZkDofDsTg7yjiHQR8HgjcTEFKzNTw8XjLPcbH5HJ4p01+6gT3mev7x0Qb/6Ee/iBhFbR6j3HfsNDclPkmjQmOkl6hd4o3z+7nQqDPcrvDqdEiUgWKwCuBzmxwhU2imSmJhX0U4UkvYV2lzz51fo/vARf6fL/4gv3D6IIlt0oy2j3E+Wv/+DtmPDvZ1yP7z8PwOO/+fB77Wsc1N55/skN3/rTd3yKI47JAdH/7vOmTf9+fSIes+9F86ZA7HdmZHGOd88c9Q9nrptX0YDLEkRES07STWToNaUo2wYpmMQ5pnhwBI2iGqgudnGC8jjQOyzENVKAUJ9TAmth69gU/k5UYh9y6DL5AqTBohttATKF1+Rl+pTfeBi5SvH6Y3jPEkIN3VPdIcDsdy2fbGWfCplA5R8fq4Wd/Cm+t1plPlkfgZLkWvkGZTgGK1zUT0Bg3vIv/ujOFLv/dBrOZLd54IN3X5HKkm1PyMfdUGJS+lq9zmQM8Y/dUGg5UqmTV45nLERmYNceZxplljIvHoDVO6w5jeSpPKTcPYO25msNzEalb4pR0Oh2NpXNM4i8hngA8Dw6p6RyHrBx4EjgIngI+o6tj6DfNqAzRUvD66zB4GqbCvYrkUGZrtMeL0/JwNLZmdIrMNXo6/zMtzjKXg8zo/yB3NgwyVAxIVuvyUeimiWmkRBAmBl4fNecbieZYsM2TW0E5CmpkPlKj5KSWTEXop9JVJBw9R9vPP5QuEDofDsTSWMnP+beDXgd+dI3sAeFhVP1U0cHwA+MTaD+/KiIT4Xg+BV+MANzKQdTNNyjcuCRPaoplcWuRTBsED0nlxFErGefsKiqV7qodTzTplEzI0UqM7OFwsBOYuDV8Uryi0KkBsheG20kgVXwLKXo3+4T5O/3IfQ7VpHnztIM3k78hsawXnKCeAKfJQkNT1snM4dg/XNM6q+qiIHF0gvg+4u3j+O8AjbLBx9kwXPaUj1GWQYzLIQMnwVGOMp9p/itrWMt0IykTrBSZ4CUFADGCQqU4/sTEhYPBMCd9UAIizSayNUSyqCYLHb17Mt8vsQ6i2V3Oq36uqi11p1oz9tfd2yD5UubFDVvM7s/pOtL4+73Xwk4vpvXMh9Nbang7Zq9E3O2R7y52Lf+M3nuqQORw7jZX6nPeq6rni+Xlg75U2XOteYTOLf54p4REAMJ2leLHPpJnA2iYAob8X3ysTJWNkdqL4tEWhSBJZSJ44ojNPmRul7CESIHhktgFqsbZEZiLAYjWFOW4LJSHNmlxOTPELufM7OxyOpbHqBUFV1av1AFvbXmEepXAfZa+X0FQJpUpKxLf1myRRiygdBTJCfx8fKv8IByse32wM80zz80A2Jx17qcPwEPEIvH76SkdRLKPtV0izcVSb2Kw9+77vlfGlhG/KpLbNVHQS1TZGqrMp4u345LUPOR8F/qrQ238qdDmLa5LpcOxcVmqcL4jIflU9JyL7geG1HNTVCEyFiunBkyAPmdOIqejUnNkxBF6NY3WPG+oxrzf6eBYpzPHyrg0iXl5/w6vRwxAWy7i8MWc/GapgjE9gqgSmQlm6iKXJtAS5cTYhJa8LgBU4N96nqmdEZA/wkIi8qKqPzrzpmmQ6HDuXlRrnLwEfBz5VPH5xzUZ0VSyt5BKJbSEYRAxWU6zmi22+10c9PMjtvJOP3vQ6N9/yMvar38vT597GdDpMKz6LaqfftBNBJGCg+maGOELd1hmiSqSWYf94RxRInE6QZi2MBDRMiIihEuwB9tDjH2BA9yEYvsPzyzpbVT1TPA6LyBeAdwGPXv1TjqXgFlvXBxE5TB48sJd8FvNpVf21zR3V9mQpoXR/SL74Nygip4FfJDfKnxWRHwdOAh9Zz0FeRkmzMdJs8ai9arCHm7iLt/dUeedHv0Lrf/hnfP8/fZHPn7mD8/4FXk/GyK5pnPPiRUYq3GBv59ZqF4GBigfTKTzb6luwvKVYOzW3Xh1GauytvpUu+tljh9gTlBCE7yzjTEWkBhhVnSqefx/wS1faPjB1hirz7cvZRme23mKca3yjQ/Yz39O5GHrd5+/tkP2v5V+f99o3H+/YRhb5mv1A9/4O2cHqD3fIfv/sdIfs3aVbOmT/tf1ih2wJrPti6y4kBf6lqj4pIl3At0XkIVVd3szEsaRojfuv8NY9yz+cQaRc+H5Xtzgm+BhTw5iQntIRKtJDVbvIbMZkAqNP3MRA5f8B3sOH9oScax3lG1plWE4xFp1YMPudi84uGPp4BAYSC1OJMpVa2jY3zcZ0UQv3ARClU2S2hdUI1ShfsJSAwIYoSmTzZchlshf4gojkQ4E/UNW/XO5OHI6NpAgUOFc8nxKRF4CDsMzbRsfGZgh6pkxP+U1MRqdJs5FV7SsM9rC/9Gb2Z/v4oT1lbuqa4hsX+/jy5Bs83or4Xz7/Dxj483v4f93xEp/8tQexxxv87Wc/zEtjd/BfT9/Oo+lnuLLBzFAtkk4ETrVT/s5+g3Y2TlTETx+svpN7K7diFZ5vTHPBDDNiTzLZPo6RkFArVLRCi4iGXb63WVVfA+5cmXYcS+Cqi63gFlxXSxGCexfw2CLvOd1eg401zvjUzABNb4R0lVFlngnpst30mTJHaw2ODQ7z4mQ3scRMMco30mHCrMrH4hLN7/kYpf1/y7Gvn8aqsH94LyaqozpTi7mzvKeSYckr0TVtymR0EmunyFNPPLptH/srFlU4364QZX00zBhTUsJIgI+PICSSEMmq4pwd68NVF1vBLbiuBhGpA58Dfla1sxyj0+212VDj3GPK3Fu5lb/VGi/HZ5hrEEXKeKaCtTFWm1zLDRCnE5z1jjMle3jw5DEGz/fwYqPJheyFPO4YMOLz689+mP/+e0cQuZ0488jU8KEDTT6QfYTTzRIPjUxx3jvDcPQiUXJ2dv+qCS/JU4w0jzAip1HbQvDpq95Orxygaqs8Xbi+x7MIK0oXA2jl1tn466Y0GZNhJtKzHeNfaw6XQn7lxkPzZL/0Smdbqe80H+yQLeYTfutDz3TIqv0XOmSf3H/3vNfxb3T6qs98tfMG4Ef/vDMx5bpGZ/swg+mQrQVusXX9EJGA3DD/vqp+frPHs13ZUOPcFaR8YO8kJ1/bz8sL3vNMjZLfS5I1iNM21/JJZ9kUE9EJJjjBab4FTchsqyMb74/i3+TBcY++6u3c3/U+buiKuffmF7jhg49x6du30P6b7+GlqZt5PJxmOJlrRDNGm08zyneYuVAYqXGE2zjm9XLJxjytrwFQkzoBIXXtplt7sKI0ZZqGmWYiOUsjeo0V+Jwd68RyF1sdS0fyRZLfAl5Q1V/d7PFsZzbUOLdSj+fHuxjTKYypg9rZdGgjAWmxqDavV99VsHZB5IVe7uk3k0k4Q2rbnG8pnoQ8c/o6/EdSzo8MUvZgX9njzuZbOV+9jktylvPNx+eE3CmCj+d1EXhdWCzTaUbEwv6CxXEkQ7FYLEYNFb8X5TpULa349WVoy7GOuMXW9eO9wI8Bz4jIU4Xsk6r65c0b0vZkQ43zhXSKf3/pIQRDPTxAYKrUzQCCx4X4Bdrx2SJS4mqzTAFMHlOhzQXvzXzOI/AH8b3yvHe/mT1DOF3lq5Pd1F95E3v9Ku8ezLiuHvPhSovecsR/O3cHv3zuDM3oxOznAn+QA+U7KWmZNk1eYxoEfPUxGDx8jAqRRDRMHvpl1GAwHLW30uNVMAgPxb+5Sg061gK32Lp+qOrXyX+kjlWyocbZasR0+3XK4QG6gn2EUqXb5t01Rk2FNhSFh/KOGKozro3LM2nBy2fEaq9QI0MQ8fBMiC8lRC77QKezPJFxBItVyw3ZO3gXvZS9jIHqNPsGLrF/vJ+6v4d2MjKb6FINBum2vfjqMW7GiKWFh0+gpfnnVzSAzYcRgEKJgB4/Xxx0OByOpbKhxrlkeriucjd120XVlvERwsJ4BvJ2xmvHqGiVXs0Xhs6Z80wyTJxN007z9Oyy30NgqrSyMdrJyLyCQ8aUCbwaRgJKXh0jPgF5zYtME2KaWE2JsmmSrMEF/yTfvNhHb9jF/vE6Q+cOkKjwk/1vR+Tt9Icp9SDh5HSFx0YypogAiLWJz+XCS21pgkAsManm26RECIYpaVJNg3U3zb17xrjvpz87T/b+x451bPerf/lTHbI/He1c6Hs57lwbq3udleRG4/kLgMPfuL1jm66B8Q7Z1z7ZmVR6+6fe1yFLvKhDVoqDDpnDsdPYUONcIeA2c4jAF4I5i/BWoW57iGw3XYHhQAVE4MXJ6zhje5n0xhmVUyiWXu8AddvDqH+eNGtj9bLvtxL0U/f2IHNW+MvUKWmZjJQpLIm08zA5jZhMzvL3YZlyVGOgNUCPlHlzr8dHbnqVgb4x+g9doDw0xvOPfBevTN1EO8qwYkltlKeOS+5bvhx2l5CRYMlmi+u3pUnTVjdMxw6HY2ewsTNnT7ihSxgoZfSHCZkKzdQjUWEk8hiPDZ7kTozMQmItCTGZJEWt5IyG5m6FRnapyMizSFF/ObURUeHzzQqjHck0vpQIpcqgPUCgAVVTplLy8UQoi8EYiERJrMUTKIUxYSlGU490qkYzLtFILU0iUiIyTbCakUo+q1MsVjOsJmTFouRMedBR35CatEMXDofDcTU2PJTuH+wf5pbrTrLvtldJm2Umzw7RbpV56fQRjk/0MpV6nGl6tFJoaELLNIm1hdWE1EY0k+G8q4imhQE0GCkj4pNkDTIbo2R570BNZyM2ukrX8+7SLeyvCG8fmOL2PaeI0oCRRp1GEvLESDfPTSglo9RrDUq1FlGzTGuyzoWpboazJhe98zTtGIltkmSNoj+hnedamWHGH96KzzKyTrG6Dodj57Khxlk1D0wzxmJKCZ4VwmobMUpXqU1vGCMSMBkYPBH64xI220NLeql5vSR+xER2nthOo2rRYqEwNLmf2Sv+Mk1opiNkNi5qXrTyRUKBwChVL6VWbeLHIY2oTKb5jN1APptvVQgbMUkSYDNDYg0lAqrahTWWQCokXosom0bnuDDy2f2cxUsxs4uKAM3o1Y1Ut8Ph2MaI6sYlR3SZIX1H+R9xrFbmulpGb5hytD5FJUioldqUgxjfTymXYkQKg6cGYzI8P8NmhvGJHppxiWZcYirKoyV8z2JQQi+l5Kek1uNco04jCXhqrMy3GqMYFarki5ADQYn+kmAV2hlkqkWfQKXsCb2hUPbgpq42eytNGmnAWHGsspcRGktfuc2+rgkEJbUeVoVGVGI8yltXBSbDiNJbbtFTbWCMctdf/9W316s0Zeh16cKqdE/8d6Md2w38vztn+dF1d3Xu7/e+0CHz9nXGdv/Bv/kn814/cKLzAvR+vzNq7X+763iH7C3/385mBGa88xzSI50ttCp3fn3ddAszKcad2Y+7gwxVXbc17d2tW4Bs0e/uhs6cp/USj7R+i8eyI/S1ruN6eyMf2tPHvnLMW/eeZf+Bc1T6pqjdfAapCewdwPb2Y6vdpAPHwGYcPvUsZvQSjEyQniuBNYifgSimFiNdgraU1it7iSZr7Hn8HUy/0cd4nPEq52nIBC9kbdJGvqjnERBImaPZMfaHZS7GCY8mefnJe6K3cHuPz75Km3fuP0O1FNHbO0651qT3xtN47+6DsIQ0pyFN4fwo8ak80sSrROBZvKMe6U13gDEustbhcCyZDTXOMyRZgylzntN+iadGb6I3rHChfZQDw/up+Ql7vzVN4KfUSm1KYUwQpISlfEbWbg2SpPuIohJT7QqqQqa528A3GaGfkllDlAYkmeFCq8q+MoTGZ7jZRWJi2kzRTPOqeLk7xOe0X6IZDzBpJpjKziMYXm5N0UzrDJXrXGyXKHuWypmUkrHsfeFWDn1zGM+zZNleVIUk9YniEKuG6ahMZg23HXuVPW9/Or+AOBwOxxJZSrH9RTsbiEg/8CBwFDgBfERVF6+Cv4A0G2UqG2daTnJaHkeaPt5kCSN+PpuVoPDV5rc6gsGIh2ComhIlrczsiVRSWjpBoq3ZiInQq3O9vY1uKXNbd8jtPS1GooDz7RqpZozqKVrxmXljmpLXeYMAJZlNfnlcTvPtOKQSDzLUuBGDockkMU1CqlTpwZD7lQ2GAdvH3qBMI8v4Ds8znQ3z0RMf5P43jhB4GXkDGYfD4bg2S5k5L9rZAPinwMOq+ikReQB4APjE0g5bdLrWbLYzydJKiAoTXi+eqSBiMOJjNSVJp/KaHJqipBipEVTLdDPI3uggjdSnmRkim5BITKYRCwsrqWYo82t1WG1gswZNtYyXqwiGRnqRNGsg4mMkmB2H4DEVHCZKDtOUJsPR86TZKCfMBzg50YdvFq8XIiKfAT4MDKvqHYVsxRc+h8OxM1j2gqCIfBH49eLv7jlNXh9R1Tdd47OrdvznBY0WXlOKcLYi3lnw8LwqRgK6wv30c4BIWoymJ4mzBkk6gdXGko9pTBf18AAiHq1kNK9+h53TzTtPKzemTOh3kdmYOBlGSdlTexe32TfjieHh5n/scPyLyPuBaeB35xjnXwFG51z4+lT1qhe+tV9U6dyXMZ3JNHmN6+XTXe78qjTizu40cxv3zlAr3dD52ehltyC4brgFwfVlDRYEF3Q22Fu0pAE4T+72WHeUdF71uWJk5EYZRAJQS5JeApQoOcslvr3Kg1oyTTFFNiBQFOq/XJ5UAZs1Ojq8jLRe4lvBcF4TZLFdqz5a6HUu95H3bQT4HeARlnxX4nA4dgJLNs4LOxsU5RYBUFW9UjeDjWlHo0DelQSiRZNCVrf3hCgdR8SQ2XY+Y+64QCyO1Yg4XfbsclMufA6HY+uwJON8hc4GF0Rk/xy3xvBin924djSX/dhrvmeNSbOZeNvlnYJqmzTrLN6z9M9v9oXP4XBsBtfMK75KZ4MvAR8vnn8c6CwztqNYUQftlX72QnHB41oXPlV9x3r6Wh0Ox+awlJnzop0NgE8BnxWRHwdOAh9ZlxHuTmYufJ9i0y58nXcgK138W4zJ9ksr/mzDpcFveSSPg30COKOqH97s8WxHrmmcr9HZ4J61Hc7uQ0T+kHzxb1BETgO/iLvwObY/PwO8AHR27XUsiU3JEHRcRlXvv8Jb7sLn2JaIyCHgB4FfBv7FJg9n2+JqWTocjrXm3wE/x1I7NTsWxRlnx45FRD4jIsMi8uwcWb+IPCQirxSPfZs5xp2GiMxku141uUBEfkJEnhCRJzZoaNsOZ5wdO5nfBu5dIHuAvOzATcDDxWvH2vFe4IdF5ATwR8A/EJHfW7iRizS6Ns44O3YsqvoosLAg9H3kWZcUjz+ykWPa6ajqz6vqIVU9CnwU+BtV/dgmD2tb4hYEHbuNJWdfuiQfx2bijLNj13K17Mvi/Q3Kbt2ZqOoj5HVhHCvAuTUcu40lZV86HJvNRs+cL0HWyB+3NYOs7ByuW+uBzOESZDNN+FY6vq3Ecs9hqbpdafbljH53gm6Xysy5ruf3FuZ/dxc7/maxUcdfVL8b2uAVQESe2O4rtFv9HLb6+JbCWpzD3OxL4AJ59uWfAJ8FjlBkX6pqZxfZdRzXdmGzz3W3H9/5nB07Fpd96djOOJ+zw+FwbEE2wzh/ehOOudZs9XPY6uNbClv1HLbquNaDzT7XXX38Dfc5OxwOh+PaOLeGw+FwbEGccXY4HI4tyIYaZxG5V0ReEpHjIrItCs6IyGER+aqIPC8iz4nIzxTyLVfdbDvqF7ZP9bjtqt9rsdn6v5ZeRaQkIg8W7z+2SLf61Rx70d/3gm3uFpEJEXmq+PuFtTr+VVHVDfkDPOBV4BgQAk8Dt23U8Vcx7v3A24rnXcDLwG3ArwAPFPIHgH+9yePclvotxv5+4G3As3NkTr+7QP9L0SvwU8B/LJ5/FHhwDY+/6O97wTZ3A3+20f8vGzlzfhdwXFVfU9WYvJzgfRt4/BWhqudU9cni+RR5652DbL3qZttSv7BtqsdtW/1ei03W/1L0OncsfwzcUzSeXjVX+X1vOqsyzsu8zTsInJrz+jRbRAlLpbidugt4jGVUN9sgtr1+F+D0u7lslP6XotfZbVQ1BSaAgbUeyILf90LeIyJPi8hfiMjta33sxVixcS666/4G8APkt/n3i8htazWwrYaI1IHPAT+rqpNz39P83mfNYxJ3qo9zuayHfp1ul856fb+3Elf7fQNPAtep6p3AvycvAbD+rMJX8x7gK3Ne/zzw89fYXnf538W19MUt2H6zz22z/9ZNt06/KPDSevhVcXZBucJ3dzW1NRa7HfmuhRt1Fiz3VnHI7c6ilbeuxKwvDkBEZnxxz1/5I063S2QFuoXdq98Mll69b7k8nj/sVt3Clb67674gqK5X2Eq5pi/ONclcMbvNf7wWfGo9dlr4kB2LsBrjfAY4POf1oULm2CDchW99cRe/y+jyyqo6f/4asBrj/Dhwk4hcLyIhefzhl9ZmWA7cxW89WZJu3cVv+ey2QIH1ZMU+Z1VNReSnga+QO4w+o6rPrdnIHLMXP3LD8VHgn2zukFbGQPWuDtlI8+83YSSz7BjdbkFW6M93LGRVxfZV9cvAl9doLFuImfh23bQRuIvf+uF0u66sMFDAsRDXCWUWQfAQKeF7XQAk2Tiq7U0b0c69+G0+Trebi7rO5tfEGecCwQPx8bwq1WAQIx4T7RbZJhpnh2Mb4tZK1ohdapw98nULg5ESIgZrY5SEzLaJsjxBSLHkd72WzXRxOBzbCOfPXyN2pXE2pkrgdeGbCnV/CIPHSPQqcXoe1YRW3EQQEB9jqqhG5DVZHNfisQ98f4fsu/72Kx2yqU/s6ZB98P+6t3N/rd9dm4E5NgTnz187dqVx9k2NWjBESer06z4AJr3zxCnkM+QMRQi8Psp+L5kmZDZG1ZJmU7mxLrZzOBzzcf78tWEXGmePW8IP8O7aIHVf2VPOsMBnL7yLp3h1ditj6vxw9Uf57qGUspdRDxKmk4A/PePzHZ5nIj1LI3oN5+5wOBzrwa4zzoJwvdfPdw9NUw9i9nVNkmQe37x4gKfmbOeZMu8aTPnhO56mWm9S3ztCa6yb83/9vYxduh71Lc3oJIrLPnU4HGvPrjPOAM0sYzQOGYlKvDTRTWQNgYG3VP8xl+Qs55qPk9k2T4z4lJ9/Mzd0T3AXLwLw3n3DHKp1Mx7fzoXWnaQqpAqphUYK06llMk14znyHqeQ8UTqGtVObfMYOh2O7seuMs6KMaZM3Gj2cayoPp4/RSsf5R7V7+V8OVPn7sbfwW/GrRMk5vjD9h/xpq8b3l36In6seZt/QRe7+R3+Bd5MHvg+lMlgLExPQSohf72HklSOcPbeP33j2nTytl3jDe47x1nPsVPfHOyofm/f6917q6djm82/tbD/3+lPnOmQPvdJ5F1I6dUeH7N/e/94O2f9+4j9cdZwOx3ZjVxlnwUekRCQRYxFcShIm4jdIsynaZcUTRQCRvORImo2RZmOc9aY5N9mDJ5ah0S4q0yNQDsHzc+Nsc8MrfkZQjqiU23QH0Nuqc8HUN/GMHQ7HdmXXGGcjNfoqN1MxPVzSs/xFdJK2nSDNJlDN+Fb2MuMnj3GBS/n2po7aFkrKs9nf8n+8/D66Ocjep4/RHQoCBEXZKE/AF6UntAyVEmJrqPnKHd1lxqeu4wKP4yI7HA7Hctg9xtmEDMgh6lkXr/IUE625dViE89HzTAUXSW0bVYtnyqQagUIrfoMn+YN809bCPQu+14tnKtwYvId3lPdRC2AgtNTKlu7J2gadocPh2EnseOMsEmKkQi3cz1HdT0/oczHbx8S8IllKmjVoAKqWzLZQLJ7pQsSgalFNUU2x2iafBXuIBAgeVlM0azAWDnO2PUAQCSdFUFVOeK+RZxg6HA7H0tnxxtkzXdTCvRwwb+LNPT79YcYbFw5xCmHuIl1mJ8jsTF9HBTxq5X10eftIiWhnE6Qa0Y4vYrWBSEDo96NqSbJRrCaMJq/zYhiQasRI+9XZhJWdshh4oPY9HbKP7a/Me/2X5zoX9f5+otohO3U86JANH/t6h6wvuK5Ddrbxn646TodjJ7CDjXNePyPwalRNH4GGNFLBE49IkmIbwUgVxIDms1slA01BDFYtGQmZJig2n0HP8R3Pvta89kaatWjaMVKNNr2incPh2N5c0ziLyGeADwPDqnpHIesHHgSOAieAj6jq2PoNc3kIPuXwAKFXZ8hczwG7j4iEv2weJ9JpRpPXAcX3+jhSfjc1rRNLTEZKWxpMpGdIbUScTXIpuVQY5tyg5zPh/DHJRmGOwc6yCcbax3NZsZ3D4XCshKW0qfptYGFFmgeAh1X1JuDh4vXWQQwlr5uaGaDX9tHnh/h4nGk/ybnGN4iSswD4Xo29dg8HpZ8Ddoh9di/9di8Vr4+S1421MZmdwNopVNvFTHjGRaGoxkWGoBaSFGunsNpgp7gyHA7H5nDNmbOqPioiRxeI7wPuLp7/DvAI8Im1HNjykdlC+YJHlE0S22lSP2LcDtEw42RZE4BScICucD8BZS4wzAg+ZalS0pCKlrhJbycl41Slj/H0FIKHJ/NVlWpEmuVuC2N8BEOSTZFmkwiCMTVEfDLbWLF7Q0ROAFPkK5Cp62XncOweVupz3quqMyle54G9azSeFSKAIfB66S4dJLURk9FJrJ2mGZ1ieHa7DBAGwhu4XW9lTJs8b79OkjUYLN3MIAcY1B5urJfwBOrTt3DWH8RXn0ADzJwbjYbXYMLPY6LL1PHxGfZeZ6LdQvCohnsITJXp+AJxen41J/e9qnppNTtYCT/a81Mdsq+nT3fI/uTM/EiUqum8GXtHb6VDlowf6JDt8zplk3Smvv/iLf9Th+wnX/zPHTKHYzuz6gVBVdWrtZlZ315hXt5cyuvBMyVCr4bBKwY2UyB/ZgFPMFLDmBAjHtM2omEa2CwPkfOlRN1WCcSQWEgAqwoCgsHDwyDYwl1h1GCMh9UMK5ZUU4x4GCnhmQq9/mHq2kPqt4nTCzg3h8PhWA4rNc4XRGS/qp4Tkf0wZ3K6gPXqFSb4eF4XgdfFMf+dDGgvIzLORU5hNZkXVQF5xt/B6jupaw8jeoYn04dQtViNMFJiX3aQmyt1mplyotkmImVamiTEIFDSEIuSSEpGSiRtrGZkJEzrpWJfGbVwHz3+Qe4p3cq+ivLwSJ3HOc4KMwQV+KtCb/+p0OVlHbgmmQ7HjmUpC4KL8SXg48XzjwNfXJvhLAPxMVIi9OoMaC9DQYm61rBkRXup+XimTJ8dYsD2kWlClJwlTi+iahExVAjpDvOU7ElajJpxWtIklRSLLYrrQ0ZKLHEhy/9SG5HYJkY8Qq9Ot+3jQEU5Uo0YMJ0xvsvgfar6NuAHgP9ZRN4/901V/bSqvsP5opePiJwQkWdE5CkReWKzx7NTEJHDIvJVEXleRJ4TkZ/Z7DFtV5YSSveH5It/gyJyGvhF4FPAZ0Xkx4GTwEfWc5DzxoMP4lMND7LfvwUPnxEdZySFUXOeqfhcnuFXhL6Vw0P0B9cDcIGTnBfLVJS7y42U6SkfpSzdjOs0T0/ApDQ5JS+SZE0CU8WnREyTyGvnroziepZJgtUMwdDj7SPQEnXtpttW6fdDesK8SL8nAYKsyKmhqmeKx2ER+QLwLuDRVSvRMcOm+PN3OCnwL1X1SRHpAr4tIg+p6vPX+qBjPkuJ1rj/Cm/ds8ZjWQJ5RIYxZfqCw9ysh0hU+Y58h/H4DdKsUWT5XTaF/cH13CW3M55FPJ78xTz/r+dV2SvHqNkaF7yznMyeIUonieLzKEop2EvZ7wWgyRhGPMrSTSAlEo3ISAgoM5DtoU6FwSBkX8VQ95XeMKHspQQiKztTkRpgVHWqeP59wC9dXTfhPMlS+x763kCHLNPOy0k3gx2yXn/+Me/q7/zcv73wVx2yYJE7ii4z1CG7r/6mzmOG4x0yx9agCBQ4VzyfEpEXgIOAM87LZJtlCCpKhmpKy04woi0yLG2dnK2HIRIABsFDxEexjGYtJmUaW8ymfW+AStBPxevDVx+LJdYmUTZJmuWVjQQp/MgJgsk7dGtGxDQpUd5XUBMwkEqGVaWZWUbaQhQIQ6U8PTm2l10iy2Qv8AXJjbsP/IGq/uXqdegouKo/37F6ihDcu4DHNnko25INNs6C4M8xVjMRFUtHNTeMo60XeNI7XRQqaqCa4HvdlII9eOJT8frwCGjYEb6dfAWrCda2MVLlttIHeWd1kOlEeTWeZNJMMJ1eJIrPgxjEVGYNcqYJnuSGVrFMxqfJsiYiPsaE+KbCdLAHXz3Oc4HJ9CLd6RCZHmWo7DM8W69jeajqa8CdK/qwYym8T1XPiMge4CEReVFV57mM3ILryhGROvA54GdVteNH4HR7bTZ+5iwG0ZnZpGH5UQxFd2zNSNKZ5I4Z14Gh5NXxpUSNPnx8pnW4cGUYRAI8U6Ff6xyoWC4aAzF56rZGKCmiPsb4zKyVqlosFk9ANSOzbaw2EQJE85m5FUsqGQ3GGY/eIA6aTCZHKHuGeLaOh2MrsRR//npFGu10JL99/Rzw+6r6+cW2cbq9NhtqnEUCSsEe4nQCtdML30XwCqO90GDnSSY582fbgk9X+QZq3iC+lAi0hMEQEGLUUPMGSUuX61x4EjAikzw1NsBY1uJ1eY5WMoaqxfcGEMldIlCUD9UYK+lsaF4t3AdAj3+Q/dlhAKZ1mmmZJNbmvBA+I1DREqVgL1YTktUlo1yVAX+I+3o/Ok/2VKOz3Mknbur0gf/kK53JJWNJ50Xl9qAzSWRPeX7Az8PDndmQR8xbOmQvx53rmh8bfF+H7JHJzvW6TDt938th+f58x1KR3A/3W8ALqvqrmz2e7cyGGmcjPlV/gMxGHU1Pcx9xCUhQ7ZxNy4xrYUFBIZES+71b2G8HSbDEpCia+4GxdNGPH5RQLJkmWDLG5CJTTDJtLjHafAHViMAfpBoMolgS25r1N6umoAarCZ4p0eMfpEYvh7P9vKmrRGzhyWbKuAznxfkXUCGgFgxhNWN8HY2zY1k4f/768V7gx4BnROSpQvZJVf3y5g1pe7KhxtknpF8O0fYmSNKL894zpkbo9xCYCjVvEMEQ08RqSmKbtJLRoth9WhQb8jBSxvOqtKXBqJRJJCaijY9P3XZRokSoAVWpkxAzbSZINCLRNjFNUhtRCgaxNqXs9xB6dazm9YgzTbBZgqoFLDPBC1LMytsknG8FxNYybkZpZmOoZoReNyVTp5lZJmJDgiWQCplzb2wZnD9//VDVr3PZz+hYBRtqnKsS8hbvCJG0aEYn5rwjdJUOs9fcwHW6l+8eNFQ8y0jkM50KJ6ZTvuU9QSsdpxUX6dZeN92lIwBcSl7ngr5YzMhjSkEfd5oPMOhVKHuG0AiN1PJqZpg0YzSyERrxOUp+P9cFb6ekZTJSEmJiE9OUsTwaw8ZktFFNir8yRjwqWYULZpjn05OkNqKdXMJqi2p4mL3+zQSEnNdJLrU9WtKiRh9WLC6g1uFwLJWN9TkDoRGCrFREbWRQLAx6ElDSMlXPYyBMqPkpqoIvHl2+R5hWSUwLER80r2ERSAXF0tQR0qyRz6o1IrM1rClqYAAlD2IrSJb7R/PIjRglo6JValqmRTSbYBJTAsndMIIBPJQ8JdwWLpc20zTic1gbz7auUjICQnz1aUlzdv4QELrSGg6HY1lsqHGe0hbfSF9ExbK/9h6adoyJ9suoxkxEb9D2J7kgQ5w7fTMBPgktElJapoUnATUzQLnSPZuerVgEw1B4Mx4BvvqUtEwqKWfkDU5omyAr42clMhKajJFlM5mDQ4SmzqSME0kZX3088s97MkhGihcERH6e7p3YvNxo045xUqZpJiNktgFqEfGAgDRrc968gicBBh+DxwAH2Wv7EYSX1lG3vsBAaf4V4Mf7O6vB3f/sZztkbyvd1yG7qzfskCXaebf64NTj88dhSh3bHLRHO2Q3h+/vkP2Hi3/QIauF+ztk/+pA5wLjb1zoEDkc25oNNc6JneLU9FfZU3snN9nbuWRGmZSTqMak2QhpNkIjepUL/B2z2YBSohT0MRTcSIVeylrFV49pM8UoZ/EwDGX7qFOhbny6Q4/ROOVryd/Tit9gJtJDxMP3ejASEHq1PA5aAlpMEkuLGj3UbIBPiZLmBsYzPi1TJyMhNXlG4GRytshEbBUZeIKhhDEhmW0xHZ1BxOB7NYz4DHmH6Q9CBFmkc7fD4XAszsYuCJoq/ZU7OaA30GVCPDtIVnn/bGuoVnKpcBM0AQVNsUBmYxKNQCDQvKuJLXr6GfGoUqbPDwmNUDJCyRiMBMwPwctlnslnhIltkkAx64XEtGl7dYDZUDqjBqMGJGBuP1hrY/KQvnz/eRnSEpaoeM/HiE9gqvRpDwerJvdwrCwfxeFw7EI21Dj3mjo/VHknCliFsufzPcWi3rdH38Sz/ktMZcNMtl5GKaIyNCXNGjTsCIGpUJba7Mx2xq2xJyixv3I53radGYKsMmtkoYix9roITJVWNkacTgF21jUxJfns9/L2Pn2l6+llLygIZRJKjNi0uHjA5Rl5jdDUaKWjWB1HKBGaOlXTx/WlCu8YmMITzdsSOBwOxxLYUOMcGOVwLWMqMTRSoexB1cv9x2VjCG0FX0pFN+zLn1NNSW2EYIi8NqGEJBKR2YRMEjKFTHODr0BSxL3lcdM5RkKMBB1JJnlMtQXN9zeDEOStqCTFYPCKGbiIIa/dISB+nnUoJYwE+f7xCllAICUCA54ogeksY+pwOBxXYkONc1+5zY/c/BIvXTjAixNdjEQejwwrk9qmKS2MmML4zcdqm1YyTBtDyxvlgvh5mJtt0ZALfKOcUGn2zC4UZpqgWCrhPnxTKsp2mtnHstdNYCpYTYiyqcvRGzqTCZh31G6mI2ReUhjafHGtFgxR9nsv708MAWU8CQhNldjvxzcl+jlA1dYZiSzfutSNWefIzwN3BPzCNw7Nk5W/+msd2z18/8c6ZJ+f/M0O2ROL+Mf31N7VIetmfiW5tjQ6tumVzoXJ16WzSNmxcmeGoF2kNvf//XJP5+Acjh3GhhrnUrXFDW97juSxgOFWhZHIy8t9Jm/QEx7KXQiLks1mFGZ2ouPdS42Rea9FQrpK11PzBwilSqgVrNgivdpSkjrGeCTaxkhAphFROkVqG4hSGHmbh+fZFGN8QlPDSEDdGyTUCobLtZ0tFhRCKmAG8PHpsb2E+EymCcenvI4xOxwOx9XYUOOcRiFjJw7Q3z/G93dPcufIIAOn38pI9HaG28p4ktFn+6AKcTZNbBukWQtr21jtnJFdEc1TsEU8rMny7ih62TgLBsHMpmgDlPwuSnQV1ehyYxpn02Q2npUZyQ06AlY7O65IYaw9CbDG4uFT0Spt2xmW5nA4HFdjKZ1QDgO/S16PQIFPq+qviUg/8CBwFDgBfERVOyvtzKHZLvOd4zfxge97BPlnb+fI+Bne852/ILsY8q0/vYe/PnWY8biLfY130dSMs2aEsWCYqew8U+3XWGoFOyUjTidIbQvPlAoXhs0rz6nNk0sK94ngYcTQ5e2jTi8lLdOtdSyW14KXGItOIHLZJZJoi0RbpLZNbPMLRp6s4uGJX0SJwHhRZ6PkdVMx7jbc4XAsj6XMnBdtOwP8U+BhVf2UiDwAPAB84mo7iq3hTKNO+1IPXeMXkLgNA9143hRDvWMcGR2kNwwoewGtLKDa2MNwWmfE6yUNI1LbIrPtvPiR2qLGxlzyynaIyfv7aYq1HilRXvjI5gbTMyFlryfvuM0AASH9tpcuKVHxPOp+briD9u1cLF1OgrBYmjJFqhHGyw3xzEwcIDRVSlJHsbR1ClVLRXqoa08ekrcIIvIZ4MPAsKreUciWfeFzOBw7C9FF2hFd9QMiXwR+vfi7e04H7kdUtbOn0BwqXr/eVP5+3l3bw9v7WxyoTfP2W1+g0jeJFyaYUoIJUrxqG80M5568lXPDezg90ce3R3oYj4Vnpqc5651hPDvLVPskSgKauxd8v49aUZ6zmQyT2YWrWhYw3FC9h7d4RzhSM3zowAX6aw18k2GMpVyKqHVNEwQp5a4GfrXNyMkDPPXqjYxGZZ4dr3G+pRgBTwSr0MosiVWGyh77K5ZMhZFIaGcwVFIGy/mM/5Ov/4dvL2zGWjRtnQZ+d45x/hVgdM6Fr09Vr3rhq3j9ekP5Q/NkTz7Uuegm7/7fOmV/+ckOmVZrndudOtUhu/TwjfNe9995vGMbc0N3575+qLOa5ON3/1GH7OJ0V4esp9y5WvmB//YnHbpdS/Kaw7t17SBDdZH00DVid+sWIFv0u7ssn/OCtjN7i35hkEfwLrqat7DjwTPNP+YNvZWvN9/EXeF+eivXcah9noPvfRreeYx08CDp0XtRTTn8V/+GfU+/zPWvHKb2wm0MN2tEWR1aB8GDhjmHtaCSG7/Q66bXO0isTZrJJdB0kfrQHgN2gBt7hdt7p3jv93yTypHh/C0rSC1DhqpQq9O8615qe+6m/Ni/ofrppxi9MIh5/UaqXpXQs5SNkilMJB7tzHC4mnBjzyRpZjjTrNHMDAcqbfbVpjGi8HqnflT10UKvc7mPvKkuwO8Aj3CNuxKHw7GzWLJxXth2RuY0LlVVvVI3g86OB0orGWU4PMGzUcgXXz/C0NmD3Hj8Jg7+yTg99SkO3PR/YcKERlwji0LEs9xy+CTXRyX2VPdxqVVlIjnKSHQTqRWmUiHKoL+kDIRpPnON30w7ExopTCXKVJpxnNM0GCfShL8fSxlud5F85UP0hpfrMAfGUg1jKkHMDTf8HcGxzzF1cj+vHr+BZlwi9DKO1NpkKkTWkNjLergY+UyN9BFb4UxTaKYKA2UO1KfwZFlxzku68Dkcjp3LkozzFdrOXBCR/XPcGsNLPWicXmAkvcioPMfzUQ0weKaEEZ+94a189zd/mMGScM/+S9xy6BR9ey9y5J5vI1W4WWRORrZAZIlP9hBP1Am6Gvi9+SKdJh6aerTODTI+PMD5i0N8/tWjvNG4jqfTU3y1/VcQwR9MBB2x1SKG0NT4wFPfy609hsAoVc9S9pRj3ZMc6RljvFXlXLNGK/XxJV8pPTkNb8RNGtLmNC8SZdP0TN3Ld+2x+N5y23HlXO3CN/euJJDObtYOh2P7spRojSu1nfkS8HHgU8XjF5d+2Mt9ANMsBiAtbNcF4NVsLxNJlaNT3VQu7COKA4JKhF9tY4IUCVLEKOJn2NineaGfuFkhaJUIm+X8CJmHzQzN8S4ajSpRElDzld7QUErKgMXamFRHWayeZyQhJ8xbCKeGKHuGntBQ8ZSaXyWzwnhcYiQKaWeG0TifoY8lKeNmkpY0SdImmUZMxMrZ6W785WUILunCN/eupOL1u6KkDscO4poLgiLyPuBrwDMwG9j7SXK/82eBI8BJ8oiC0Wvs65qOf5GQUrCHwFQY8I/Ra/upaEifKeOJYAp3ipBPnDOFVpaRoXgI/hx3iyVftPNFqPuGW3oyqp7lGxc9/i57loYdYbJ9vKgu1zESQn8vlaAfweSZhgR0MUBJ8wuAlbz5a0MmiLWJJSO1EUY8AqngEVCmTk3zgkrPNP9oUcd/4XP+szkLgv8GGJmzINivqj+3Wt2uB//l9v9x3uv/34nOJKGXGku7bgf+UIfMW6QE6ZHgrg7Zyw23ILh+LH9BUPJkgSeAM6r64Wtsu4t1CyteELxG25l7VjuszuPFtOPTtIEpXln1/jzTQzkY5Ih9C7exh74wpjes0dMaRI1lksVD3ECJ0/PEC/r+zZQNFgnxTB5JYDVGNcEzFTxTwTcVasEAJa0wnB3n9ej12YiShYjIH5Iv/g2KyGngF8nvRj4rIj9OceFbpRocjo3mZ4AXgM5QHceS2NAMwc3AakSUjnPOvMIjw0N0+1VeSS9xTl8hyiZBF8ZKLw3VDGvzTtMz4XyZbWE1JbMRo4AnPq109HJxpUX3o/df4RBrfuFzODYCETkE/CDwy8C/2OThbFt2vHFWbZNmbcZbo/y1vAyYIsRuZoFupa7arDOlXFNU21ggzWY8PM4V7Nh1/Dvg54DOIPWChSG2jk6udE+/A1FUY1TbRWahsr6Gc73377gWIvIZERkWkWfnyPpF5CEReaV47NvMMe40RGQm2/XbV9tOVT+tqu9Yz3WC7c6yMwRXdTDn+F+3RSun207drlX2ZfG5XazfpS8Iisj/CfwYedmHMrnP+fOq2lmr9vJndrFu4Up2YRfNnB27DVV9FFgYQXQfedYlxeOPbOSYdjqq+vOqekhVjwIfBf7maobZcWV2vM/Z4VjAkrMvnV/UsZk44+zYtVwt+7J4f0HpAcdyUNVHyOvCOFaAc2s4dhsXiqxLllt2wOHYSDZ65nwJskb+uK0ZZGXncN1aD2QOlyA7WTxf6fi2Ess9h6XqdqVlB2b0uxN0u1RmznU9v7cw/7u72PE3i406/qL63dBoDQAReWK7h89s9XPY6uNbCmtxDnOzL8mTO38R+BOWWXZgrce1Xdjsc93tx3c+Z8eOxWVfOrYzzufscDgcW5DNMM6f3oRjrjVb/Ry2+viWwlY9h606rvVgs891Vx9/w33ODofD4bg2zq3hcDgcW5ANNc4icq+IvCQix4u6BlseETksIl8VkedF5DkR+ZlCvuUK6GxH/cL2KVC0XfV7LTZb/9fSq4iUROTB4v3HFmmIvJpjL/r7XrDN3SIyISJPFX+/sFbHvyqquiF/5JVNXgWOASHwNHDbRh1/FePeD7yteN4FvAzcBvwK8EAhfwD415s8zm2p32Ls7wfeBjw7R+b0uwv0vxS9Aj8F/Mfi+UeBB9fw+Iv+vhdsczd5p6IN/X/ZyJnzu4Djqvqa5n2h/oi8CM2WRlXPqeqTxfMp8u4OB9l6BXS2pX5h2xQo2rb6vRabrP+l6HXuWP4YuKfobbpqrvL73nQ20jgfBE7NeX2aLaKEpVLcTt1F3j9xyQV0Nohtr98FOP1uLhul/6XodXYbVU2BCWBgrQey4Pe9kPeIyNMi8hcicvtaH3sxXBLKEhGROvA54GdVdXLuhVv16gV0HKvD6Xdz2Q36X/j7XvD2k8B1qjotIv+QPMv0pvUe00bOnM8Ah+e8PlTItjwiEpD/x/2+qn6+EG+1AjrbVr9XwOl3c9ko/S9Fr7PbiIgP9AAjazWAK/y+Z1HVSVWdLp5/GQhEZHCtjn8lNtI4Pw7cJCLXi0hI7tj/0gYef0UUvq3fAl5Q1V+d89ZMAR1YXgGd9WJb6vcqOP1uLhul/6Xode5Y/gfyAv5rMpO/yu977jb7ZnzcIvIucru5ZheHK7KRq4/APyRfDX0V+N83evVzhWN+H3kzwO8ATxV//5Dc5/Uw8Arw10D/FhjrttNvMe4/BM4BCbnP8cedfneP/hfTK/BLwA8Xz8vAfwWOA98Cjq3hsa/0+/7nwD8vtvlp4DnySJK/A757I/5fXIagw+FwbEFchqDD4XBsQZxxdjgcji2IM84Oh8OxBXHG2eFwOLYgzjg7HA7HFsQZZ4fD4diCOOPscDgcWxBnnB0Oh2ML8v8HQV7Uiro9wKQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 12 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "f, axarr = plt.subplots(3,4)\n",
    "FIRST_IMAGE=0\n",
    "SECOND_IMAGE=23\n",
    "THIRD_IMAGE=28\n",
    "CONVOLUTION_NUMBER = 1\n",
    "layer_outputs = [layer.output for layer in model.layers]\n",
    "activation_model = tf.keras.models.Model(inputs = model.input, outputs = layer_outputs)\n",
    "for x in range(0,4):\n",
    "    # 第0+1张图片\n",
    "    f1 = activation_model.predict(test_images[FIRST_IMAGE].reshape(1, 28, 28, 1))[x] # x代表第几层，能显示出来的就只有3层\n",
    "    axarr[0,x].imshow(f1[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')\n",
    "    axarr[0,x].grid(False)\n",
    "    # 第7+1张图片\n",
    "    f2 = activation_model.predict(test_images[SECOND_IMAGE].reshape(1, 28, 28, 1))[x]\n",
    "    axarr[1,x].imshow(f2[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')\n",
    "    axarr[1,x].grid(False)\n",
    "    # 第26+1张图片\n",
    "    f3 = activation_model.predict(test_images[THIRD_IMAGE].reshape(1, 28, 28, 1))[x]\n",
    "    axarr[2,x].imshow(f3[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')\n",
    "    axarr[2,x].grid(False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred = activation_model.predict(test_images[0].reshape(1,28,28,1))\n",
    "len(pred)  # 7层"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x28142d61f48>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP0AAAD4CAYAAAAn+OBPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAALyklEQVR4nO3db6zddX3A8feHUtpRIWnHrLWQoa4+IEss27U6JRND5qBLVswSYh+YbiG5PhCDiQ9GnAk82BKyTJgPNpM6mN3iMCZIaDaidp2RLRrWgh0UUFtJGe1Kr6YPqEsolH724P663ME9f3rO73fOufu8X8nNPff3O/f8Pjnhze/8u99GZiKpjkumPYCkyTJ6qRijl4oxeqkYo5eKuXSSB7ss1uRa1k3ykFIpr/LfvJZno991xoo+Im4GvgSsAv4mM+/td/21rOMDcdM4h5TUxxO5f+B1Rn54HxGrgL8CbgGuA3ZGxHWj3p6kyRjnOf024GhmvpCZrwFfB3a0M5akrowT/WbgpSU/H2+2SZphnb+QFxHzwDzAWi7v+nCSBhjnTH8CuGbJz1c32/6PzNydmXOZObeaNWMcTlIbxon+ALAlIt4VEZcBnwD2tjOWpK6M/PA+M89FxB3At1l8y+7BzHy2tckkdWKs5/SZ+RjwWEuzSJoAP4YrFWP0UjFGLxVj9FIxRi8VY/RSMUYvFWP0UjFGLxVj9FIxRi8VY/RSMUYvFWP0UjFGLxVj9FIxRi8VY/RSMUYvFWP0UjFGLxVj9FIxRi8VY/RSMUYvFWP0UjFGLxVj9FIxRi8VY/RSMUYvFWP0UjFGLxVz6Ti/HBHHgDPAG8C5zJxrYyhJ3Rkr+sZHM/PnLdyOpAnw4b1UzLjRJ/CdiHgyIuaXu0JEzEfEwYg4+DpnxzycpHGN+/D+hsw8ERFvB/ZFxI8y8/GlV8jM3cBugCtjQ455PEljGutMn5knmu8LwCPAtjaGktSdkaOPiHURccWFy8DHgMNtDSapG+M8vN8IPBIRF27nHzLzW61MJakzI0efmS8A72txFkkT4Ft2UjFGLxVj9FIxRi8VY/RSMUYvFWP0UjFt/Gmt1NfLd36o7/53fOn7E5pk+o796W/13X/tF37Q+Qye6aVijF4qxuilYoxeKsbopWKMXirG6KVifJ9eYzn1mf7vwQOc+c1X++5/R1vD9HHFv1418DqHXrq67/7z5/qfIz/63p8MPMbDV9/fd/8ffOGDA29jXJ7ppWKMXirG6KVijF4qxuilYoxeKsbopWIic3L/vNyVsSE/EDdN7Hjq3rf/69DA6/zuO7d2PocWPZH7eSVPR7/reKaXijF6qRijl4oxeqkYo5eKMXqpGKOXijF6qRgX0VBf8S+b++7/yPy2gbexln9vaxy1YOCZPiIejIiFiDi8ZNuGiNgXEUea7+u7HVNSW4Z5eP9V4OY3bbsL2J+ZW4D9zc+SVoCB0Wfm48DpN23eAexpLu8Bbm13LEldGfU5/cbMPNlcfhnY2OuKETEPzAOs5fIRDyepLWO/ep+Lf6bX80/1MnN3Zs5l5txq1ox7OEljGjX6UxGxCaD5vtDeSJK6NGr0e4FdzeVdwKPtjCOpawOf00fEQ8CNwFURcRy4G7gX+EZE3A68CNzW5ZDqztG/7P+PK9xyxQ/77j/yjyfaHEcTMDD6zNzZY5dL4EgrkB/DlYoxeqkYo5eKMXqpGKOXijF6qRijl4pxEY3/x/7z7g8NvM7f/v5f993/Z+/e2tI0mhWe6aVijF4qxuilYoxeKsbopWKMXirG6KVifJ9+BTv1mf7vw7967dmBt+H78PV4ppeKMXqpGKOXijF6qRijl4oxeqkYo5eK8X36KTn7e+/vv//KVQNvY9XZnv+EIADv/aMnL2om1eCZXirG6KVijF4qxuilYoxeKsbopWKMXirG6KVi/HDOMs5/5Pq++3/2vl8aeBvv/NapvvvX/NOB/vsHHkEazcAzfUQ8GBELEXF4ybZ7IuJERBxqvrZ3O6aktgzz8P6rwM3LbL8/M7c2X4+1O5akrgyMPjMfB05PYBZJEzDOC3l3RMTTzcP/9b2uFBHzEXEwIg6+zuCFGiV1a9Tovwy8B9gKnAS+2OuKmbk7M+cyc261L09JUzdS9Jl5KjPfyMzzwFeAbe2OJakrI0UfEZuW/Phx4HCv60qaLQPfp4+Ih4Abgasi4jhwN3BjRGwFEjgGfGroI17Se3GIS9ZdPvDXz585M/ShRnXJ937Yd//G7w2+jTdamkVq28DoM3PnMpsf6GAWSRPgx3ClYoxeKsbopWKMXirG6KVijF4qxuilYia6iMa5X1vLwn1beu5/+44fTXAaqSbP9FIxRi8VY/RSMUYvFWP0UjFGLxVj9FIxE32f/tKjr/pevDRlnumlYoxeKsbopWKMXirG6KVijF4qxuilYoxeKsbopWKMXirG6KVijF4qxuilYoxeKsbopWKMXirG6KViBkYfEddExHcj4rmIeDYi7my2b4iIfRFxpPm+vvtxJY1rmDP9OeBzmXkd8EHg0xFxHXAXsD8ztwD7m58lzbiB0Wfmycx8qrl8Bnge2AzsAPY0V9sD3NrRjJJadFELY0bEtcD1wBPAxsw82ex6GdjY43fmgXmAtVw+8qCS2jH0C3kR8TbgYeCzmfnK0n2ZmUAu93uZuTsz5zJzbjVrxhpW0viGij4iVrMY/Ncy85vN5lMRsanZvwlY6GZESW0a5tX7AB4Ans/M+5bs2gvsai7vAh5tfzxJbRvmOf2HgU8Cz0TEoWbb54F7gW9ExO3Ai8BtnUwoDeHs9vf33b/msQMTmmT2DYw+M/8NiB67b2p3HEld8xN5UjFGLxVj9FIxRi8VY/RSMUYvFXNRn72XZpXvww/PM71UjNFLxRi9VIzRS8UYvVSM0UvFGL1UjNFLxRi9VIzRS8UYvVSM0UvFGL1UjNFLxRi9VIzRS8UYvVSM0UvFGL1UjNFLxRi9VIzRS8UYvVSM0UvFGL1UzMDoI+KaiPhuRDwXEc9GxJ3N9nsi4kREHGq+tnc/rqRxDfPPWp0DPpeZT0XEFcCTEbGv2Xd/Zv5Fd+NJatvA6DPzJHCyuXwmIp4HNnc9mKRuXNRz+oi4FrgeeKLZdEdEPB0RD0bE+raHk9S+oaOPiLcBDwOfzcxXgC8D7wG2svhI4Is9fm8+Ig5GxMHXOTv+xJLGMlT0EbGaxeC/lpnfBMjMU5n5RmaeB74CbFvudzNzd2bOZebcata0NbekEQ3z6n0ADwDPZ+Z9S7ZvWnK1jwOH2x9PUtuGefX+w8AngWci4lCz7fPAzojYCiRwDPhUB/NJallk5uQOFvEz4MUlm64Cfj6xAUbnnO1aCXOuhBnhrXP+amb+Sr9fmGj0bzl4xMHMnJvaAENyznathDlXwoww2px+DFcqxuilYqYd/e4pH39YztmulTDnSpgRRphzqs/pJU3etM/0kibM6KViphZ9RNwcET+OiKMRcde05hgkIo5FxDPNmgEHpz3PBc0fOS1ExOEl2zZExL6IONJ8n+ofQfWYcebWYeizZsSs3Z+trG0xlef0EbEK+AnwO8Bx4ACwMzOfm/gwA0TEMWAuM2fqgxoR8dvAL4C/y8xfb7b9OXA6M+9t/ke6PjP/eMZmvAf4xSytw9B8pHzT0jUjgFuBP2S27s9ec97GRdyn0zrTbwOOZuYLmfka8HVgx5RmWZEy83Hg9Js27wD2NJf3sPgfxNT0mHHmZObJzHyquXwGuLBmxKzdn73mvCjTin4z8NKSn48zuwtzJPCdiHgyIuanPcwAG5tFTwBeBjZOc5g+ZnYdhjetGTGz9+c4a1v4Qt5gN2TmbwC3AJ9uHrLOvFx83jaL78cOtQ7DNCyzZsT/mqX7c9S1LS6YVvQngGuW/Hx1s23mZOaJ5vsC8Ag91g2YEacu/Mlz831hyvO8xbDrMEzacmtGMIP35zhrW1wwregPAFsi4l0RcRnwCWDvlGbpKSLWNS+YEBHrgI8x2+sG7AV2NZd3AY9OcZZlzeI6DL3WjGDG7s/W1rbIzKl8AdtZfAX/p8CfTGuOATO+G/iP5uvZWZoTeIjFh3Kvs/iayO3ALwP7gSPAPwMbZnDGvweeAZ5mMapNM3Bf3sDiQ/engUPN1/YZvD97zXlR96kfw5WK8YU8qRijl4oxeqkYo5eKMXqpGKOXijF6qZj/Aau9h3ac0rDHAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 第一个卷积层\n",
    "pred[0].shape  # (1, 26, 26, 64)  26*26像素*64个图片\n",
    "# 取一个图片看看结果\n",
    "plt.imshow(pred[0][0,:,:,1])  # 所有行所有列下表为1的过滤器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x281443bd488>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP0AAAD4CAYAAAAn+OBPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAOoUlEQVR4nO3dXYxcd3nH8d+z6/VuvH7bdYjxWx3HxCpOQ51264QmQqkiaDAXDlyksSrkSqmWC1KBxEUjoopc9CKgQtQLRGWSCBfSoCCI4oqojbGQrFBwbacmtpM0Nu4a22x2Yxz8mvW+Pb3Y43ZJPP//eOfMy/J8P9JqZ89z5vwfz+7PZ+acM/8xdxeAONqa3QCAxiL0QDCEHgiG0APBEHogmDmNHGyudXqXuhs5JBDKiC5q1C9bap2aQm9m90r6R0ntkp5w98dS63epW7fbPbUMCSBhj+/KrjPjp/dm1i7p65I+Lmm9pC1mtn6m2wPQGLW8pt8o6ai7H3P3UUnflbS5nLYA1EstoV8h6cS0n08WywC0sLofyDOzfkn9ktSlefUeDkBGLXv6U5JWTft5ZbHst7j7Nnfvc/e+DnXWMByAMtQS+r2SbjazNWY2V9IDknaU0xaAepnx03t3HzezhyT9u6ZO2T3l7odL6wxAXdT0mt7dX5D0Qkm9AGgALsMFgiH0QDCEHgiG0APBEHogGEIPBEPogWAIPRAMoQeCIfRAMIQeCIbQA8EQeiAYQg8EQ+iBYAg9EAyhB4Ih9EAwhB4IhtADwRB6IBhCDwRD6IFgCD0QDKEHgiH0QDCEHgiG0APBEHogGEIPBEPogWAIPRDMnGY3AOD/zVmzOrvO5PDpijW7lN+P1xR6MxuQdF7ShKRxd++rZXsA6q+MPf2fuXvl/3oAtBRe0wPB1Bp6l/Sime03s/6rrWBm/Wa2z8z2jelyjcMBqFWtT+/vcvdTZnaDpJ1m9rq7756+grtvk7RNkhZar9c4HoAa1bSnd/dTxfdhSc9J2lhGUwDqZ8ahN7NuM1tw5bakj0k6VFZjAOqjlqf3SyU9Z2ZXtvMv7v5vpXSFUM7+5R3ZdRYduZhe4T8PltRNc731keXZdRb+cknFmu/9Ufb+Mw69ux+T9IczvT+A5uCUHRAMoQeCIfRAMIQeCIbQA8EQeiAYQg8EwyQaqLtf//WHk/XLiy27jbP3peurv3Jrsu57W+PinaOPpy9EWvu9S9lttI1NVi5O5N/ewp4eCIbQA8EQeiAYQg8EQ+iBYAg9EAyhB4LhPD1qcvkTf5Jd5+Ly9Hn4xUcT550L3V/vSK/w5aFk+Vdn1+fH6BxN1s9d6krWP/WBn2fHOHH0QrJ+/sb52W30vHikYs1G0/8GiT09EA6hB4Ih9EAwhB4IhtADwRB6IBhCDwTDeXoktXWlz03/6s78n9Dv7RxJ1ufuP5rdxtCWW5L1NR3pD0fd3fdEdozvnPtgsv43PceT9U8d/Wh2jOt2LkjWe/a/ld2GehdXrp3L/z7Y0wPBEHogGEIPBEPogWAIPRAMoQeCIfRAMIQeCMbc85Pjl2Wh9frtdk/DxkPt3vinjcn66n/N//10/nBvzX20zZuXrE9eyn9IxO+K9nVrK9Z+OrBdZ0cGk7OWZPf0ZvaUmQ2b2aFpy3rNbKeZHSm+91xT1wCappqn99+SdO+7lj0saZe73yxpV/EzgFkgG3p33y3pzLsWb5a0vbi9XdJ95bYFoF5m+oabpe4+WNx+U9LSSiuaWb+kfknqUvp1GYD6q/novU8dCax4NMfdt7l7n7v3daiz1uEA1GimoR8ys2WSVHwfLq8lAPU009DvkLS1uL1V0vPltAOg3rKv6c3sGUl3S7rezE5K+pKkxyQ9a2YPSjou6f56Non6Gfj7DyfrC19Pf1BF5w//o8x2Kop0Hj7n4rolFWuTb7Zn758NvbtvqVDiKhtgFuIyXCAYQg8EQ+iBYAg9EAyhB4Ih9EAwhB4Ihk+4+R124u/+NLvO6A1jyfpNX349WZ+8po5QhrNrKsd2Yk/6YiqJPT0QDqEHgiH0QDCEHgiG0APBEHogGEIPBMN5+lnsjSf60iuMjWe38cFHBpL1ifPnr6Gjq7M/viVZ9/2Hax4jkq4zla+OsPyvnD09EA2hB4Ih9EAwhB4IhtADwRB6IBhCDwTDefo6mbNiebI+tGl1st7zwMnsGJ0/6UjWb3o8/V54SZp4++3sOilzVq/KruMDg8n6RBXjtHV3J+t2XVd6jNO/rmKU2WHR0z+rWGv3i9n7s6cHgiH0QDCEHgiG0APBEHogGEIPBEPogWAIPRDMrLs4p33d2mR9dPmi7DZGF6X/2Zfe156sn7nVs2Os/P2hZL3T0/W3ns9f9LL22V8k6+NVXHjTvqQ3WfeRy+n622ezY0ycO5ddJ2fy0qX0ChfzF6W0grZ585L17L+zjB5yK5jZU2Y2bGaHpi171MxOmdmB4mtTfdsEUJZqnt5/S9K9V1n+uLtvKL5eKLctAPWSDb2775Z0pgG9AGiAWg7kPWRmrxRP/3sqrWRm/Wa2z8z2jSn9+hBA/c009N+QtFbSBkmDkr5aaUV33+bufe7e16HOGQ4HoCwzCr27D7n7hLtPSvqmpI3ltgWgXmYUejNbNu3HT0o6VGldAK0le57ezJ6RdLek683spKQvSbrbzDZIckkDkj5T1Wjd10kf+lDF8i//fH5Vm0kZWZqf7b998Wiy3j0vc+zheP5agMvfeX+yvuRn6fP0i87kJ8DwyfT1Au3r12W3MXlkID3GWPqxahjPXxsxG+TOw+fO40uSjyZ+J1V82EU29O6+5SqLn8xvGkAr4jJcIBhCDwRD6IFgCD0QDKEHgiH0QDCEHgimoZNojC+b1OlHRirWR3+TnrxCkrr/67pkfdXO/Jt65py+kKxPHDmWrL+/hAtFqvlUl5rV+Ok1DdOW/71rMvOIbbw1WZ4zmH8s/J13knVbuCC9gXcq/21XPcaihdltjB8/kRgge3f29EA0hB4IhtADwRB6IBhCDwRD6IFgCD0QTEPP03ccG9MNf3GyYn3B3bdktzHSM5msX1iVPo8vSV3z0v/stuW3Jetzh85nx/D/SZxLlTQ5kj+n2whzVixP1i+vS08GYmPp34cktY+kZ3Y4vil/bro9M5fHyl3p38lv7liRHePCyvT1AuNd6ftf7s2fJO88Y8n6DS/nJy2ZmzpPXwX29EAwhB4IhtADwRB6IBhCDwRD6IFgCD0QjHkDP0RgofX67XZPw8abqcm7NiTrZz+QvxZgPLPKSG/6fG1bFW+4n8icNx5dlD+H3jWc/n9//sn038fib/80O8ZsYR1z0/Wu9GcxTp7PX79Rb3t8l855+mIA9vRAMIQeCIbQA8EQeiAYQg8EQ+iBYAg9EAyhB4Jp6CQas0XbSweS9Z6XGtMHGsvH0hNY5OqzRXZPb2arzOzHZvaqmR02s88Vy3vNbKeZHSm+99S/XQC1qubp/bikL7j7ekl3SPqsma2X9LCkXe5+s6Rdxc8AWlw29O4+6O4vF7fPS3pN0gpJmyVtL1bbLum+OvUIoETX9JrezG6UdJukPZKWuvtgUXpT0tIK9+mX1C9JXZo340YBlKPqo/dmNl/S9yV93t3PTa/51Fv1rvp2LHff5u597t7XofS7lADUX1WhN7MOTQX+aXf/QbF4yMyWFfVlkobr0yKAMlVz9N4kPSnpNXf/2rTSDklbi9tbJT1ffnsAylbNa/o7JX1a0kEzO1As+6KkxyQ9a2YPSjou6f66dAigVNnQu/tLkirNxNH60+AA+C1chgsEQ+iBYAg9EAyhB4Ih9EAwhB4IhtADwRB6IBhCDwRD6IFgCD0QDKEHgiH0QDCEHgiG0APBEHogGEIPBEPogWAIPRAMoQeCIfRAMIQeCIbQA8EQeiAYQg8EQ+iBYAg9EAyhB4Ih9EAwhB4IhtADwRB6IJhs6M1slZn92MxeNbPDZva5YvmjZnbKzA4UX5vq3y6AWs2pYp1xSV9w95fNbIGk/Wa2s6g97u7/UL/2AJQtG3p3H5Q0WNw+b2avSVpR78YA1Mc1vaY3sxsl3SZpT7HoITN7xcyeMrOespsDUL6qQ29m8yV9X9Ln3f2cpG9IWitpg6aeCXy1wv36zWyfme0b0+XaOwZQk6pCb2Ydmgr80+7+A0ly9yF3n3D3SUnflLTxavd1923u3ufufR3qLKtvADNUzdF7k/SkpNfc/WvTli+bttonJR0qvz0AZavm6P2dkj4t6aCZHSiWfVHSFjPbIMklDUj6TB36A1Ayc/fGDWb2lqTj0xZdL+l0wxqYOfos12zoczb0KL23z9Xu/r7UHRoa+vcMbrbP3fua1kCV6LNcs6HP2dCjNLM+uQwXCIbQA8E0O/Tbmjx+teizXLOhz9nQozSDPpv6mh5A4zV7Tw+gwQg9EEzTQm9m95rZf5vZUTN7uFl95JjZgJkdLOYM2Nfsfq4o3uQ0bGaHpi3rNbOdZnak+N7UN0FV6LHl5mFIzBnRao9nKXNbNOU1vZm1S3pD0kclnZS0V9IWd3+14c1kmNmApD53b6kLNczsI5IuSPpnd/+DYtlXJJ1x98eK/0h73P1vW6zHRyVdaKV5GIpLypdNnzNC0n2S/kqt9XhW6vN+XcNj2qw9/UZJR939mLuPSvqupM1N6mVWcvfdks68a/FmSduL29s19QfRNBV6bDnuPujuLxe3z0u6MmdEqz2elfq8Js0K/QpJJ6b9fFKtOzGHS3rRzPabWX+zm8lYWkx6IklvSlrazGYSWnYehnfNGdGyj2ctc1twIC/vLnf/I0kfl/TZ4ilry/Op122teD62qnkYmuEqc0b8n1Z6PGc6t8UVzQr9KUmrpv28sljWctz9VPF9WNJzqjBvQIsYuvKW5+L7cJP7eY9q52FotKvNGaEWfDxrmdviimaFfq+km81sjZnNlfSApB1N6qUiM+suDpjIzLolfUytPW/ADklbi9tbJT3fxF6uqhXnYag0Z4Ra7PEsbW4Ld2/Kl6RNmjqC/wtJjzSrj0yPN0n6efF1uJX6lPSMpp7KjWnqmMiDkpZI2iXpiKQfSeptwR6/LemgpFc0FaplLfBY3qWpp+6vSDpQfG1qwcezUp/X9JhyGS4QDAfygGAIPRAMoQeCIfRAMIQeCIbQA8EQeiCY/wWxlGaTNJ9RogAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 取一个图片看看结果\n",
    "plt.imshow(pred[0][0,:,:,57])  # 所有行所有列下表为57的过滤器\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x28142d2df48>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPUAAAD4CAYAAAA0L6C7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAI0UlEQVR4nO3dTYhdhR2G8fftZEw0Flw0C5sJTRYiBMEEhlTILkWMH+jWgK6E2VSIIIguXXQrbtwEDRaUiKALCZYQakSkNjqJieRDS7AWY4W0FdG0NB/6djF3kUom99ybc+6Z++f5wcDcucO5L2GenDtnhjtOIgB1/KzvAQDaRdRAMUQNFEPUQDFEDRSzqouD3uDVWaO1XRwagKT/6t+6mAu+2n2dRL1Ga/Vr/6aLQwOQdDh/XPY+nn4DxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFNIra9k7bn9k+Y/vprkcBGN/QqG3PSHpB0r2SNkvaZXtz18MAjKfJmXqbpDNJPk9yUdJrkh7qdhaAcTWJer2kL6+4fXbwsf9je8H2ou3FS7rQ1j4AI2rtQlmSPUnmk8zPanVbhwUwoiZRfyVpwxW35wYfA7ACNYn6I0m32d5k+wZJD0t6q9tZAMY19MX8k1y2/bikA5JmJO1NcrLzZQDG0ugvdCR5W9LbHW8B0AJ+owwohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKGRm17r+1ztk9MYhCA69PkTP2ypJ0d7wDQkqFRJ3lP0jcT2AKgBXxPDRSzqq0D2V6QtCBJa3RTW4cFMKLWztRJ9iSZTzI/q9VtHRbAiHj6DRTT5Eda+yR9IOl222dtP9b9LADjGvo9dZJdkxgCoB08/QaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoJjWXngQmHYH/n6s7wmNbbvnP8vex5kaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYoZGbXuD7UO2T9k+aXv3JIYBGE+T1yi7LOnJJEdt/1zSEdsHk5zqeBuAMQw9Uyf5OsnRwfvfSzotaX3XwwCMZ6RXE7W9UdJWSYevct+CpAVJWqOb2tgGYAyNL5TZvlnSG5KeSPLdT+9PsifJfJL5Wa1ucyOAETSK2vasloJ+Ncmb3U4CcD2aXP22pJcknU7yXPeTAFyPJmfq7ZIelbTD9rHB230d7wIwpqEXypK8L8kT2AKgBfxGGVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxYz0aqIV/XXfnX1PGMmqE2v7njCSDb/7U98TGrvnl1v6ntDYX/KvZe/jTA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRQzNGrba2x/aPu47ZO2n53EMADjafJyRhck7Uhy3vaspPdt/yHJnzveBmAMQ6NOEknnBzdnB2/pchSA8TX6ntr2jO1jks5JOpjkcKerAIytUdRJfkiyRdKcpG227/jp59hesL1oe/GSLrQ8E0BTI139TvKtpEOSdl7lvj1J5pPMz2p1S/MAjKrJ1e91tm8ZvH+jpLslfdrxLgBjanL1+1ZJv7c9o6X/BF5Psr/bWQDG1eTq9yeStk5gC4AW8BtlQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0U0+SVT0rbtOt43xOAVnGmBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoJjGUduesf2x7f1dDgJwfUY5U++WdLqrIQDa0Shq23OS7pf0YrdzAFyvpmfq5yU9JenH5T7B9oLtRduLl3ShjW0AxjA0atsPSDqX5Mi1Pi/JniTzSeZntbq1gQBG0+RMvV3Sg7a/kPSapB22X+l0FYCxDY06yTNJ5pJslPSwpHeSPNL5MgBj4efUQDEj/dmdJO9KereTJQBawZkaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFinKT9g9r/kPS3lg/7C0n/bPmYXZqmvdO0VZquvV1t/VWSdVe7o5Oou2B7Mcl83zuamqa907RVmq69fWzl6TdQDFEDxUxT1Hv6HjCiado7TVul6do78a1T8z01gGam6UwNoAGiBoqZiqht77T9me0ztp/ue8+12N5r+5ztE31vGcb2BtuHbJ+yfdL27r43Lcf2Gtsf2j4+2Pps35uasD1j+2Pb+yf1mCs+atszkl6QdK+kzZJ22d7c76prelnSzr5HNHRZ0pNJNku6S9JvV/C/7QVJO5LcKWmLpJ227+p3UiO7JZ2e5AOu+KglbZN0JsnnSS5q6S9vPtTzpmUleU/SN33vaCLJ10mODt7/XktffOv7XXV1WXJ+cHN28Lair/LanpN0v6QXJ/m40xD1eklfXnH7rFboF940s71R0lZJh3uesqzBU9ljks5JOphkxW4deF7SU5J+nOSDTkPU6JjtmyW9IemJJN/1vWc5SX5IskXSnKRttu/oedKybD8g6VySI5N+7GmI+itJG664PTf4GFpge1ZLQb+a5M2+9zSR5FtJh7Syr11sl/Sg7S+09C3jDtuvTOKBpyHqjyTdZnuT7Ru09Ifv3+p5Uwm2LeklSaeTPNf3nmuxvc72LYP3b5R0t6RPex11DUmeSTKXZKOWvmbfSfLIJB57xUed5LKkxyUd0NKFnNeTnOx31fJs75P0gaTbbZ+1/Vjfm65hu6RHtXQWOTZ4u6/vUcu4VdIh259o6T/6g0km9mOiacKviQLFrPgzNYDREDVQDFEDxRA1UAxRA8UQNVAMUQPF/A/qxNOjPrQNZwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 取一个图片看看结果\n",
    "plt.imshow(pred[3][0,:,:,1])  # 所有行所有列下表为1的图片  # 第二层 maxpooling\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAUkUlEQVR4nO3dfZRcdX3H8c93d5NsshsCSTYhbjaQwIaYcFTMgrS0KAg1wTbRYilbPWChplbokz3GgJYKnmOl9tij50RxqRG0NZjiA4tNjZqCqQ8Iy0FzEmJgkyjZhDwDkmey++0fM5udO3d25+7m7s7sb96vc/Y4v9/8MvPNl9mPN/fOvdfcXQCA0a+q1AUAANJBoANAIAh0AAgEgQ4AgSDQASAQBDoABKJooJvZKjPbZ2ab+nnezOzzZtZpZhvN7M3plwkAKCbJFvoDkhYN8PxiSc3Zn2WSvnjmZQEABqtooLv7BkmHBliyVNJXPeMJSWeb2Yy0CgQAJFOTwms0StqZM+7Kzr2Yv9DMlimzFa+6urqF8+bNS+Hty9vTTz99wN0bSl0HgPClEeiJuXubpDZJamlp8Y6OjpF8+5Iws9+UugYAlSGNb7nsktSUM56ZnQMAjKA0Ar1d0k3Zb7tcLukVd4/tbgEADK+iu1zMbLWkt0maamZdkv5J0hhJcvf7JK2VdJ2kTklHJf35cBULAOhf0UB399Yiz7uk21KrCAAwJJwpCgCBINABIBAEOgAEgkAHgEAQ6AAQCAIdAAJBoANAIAh0AAgEgQ4AgSDQASAQBDoABIJAB4BAEOgAEAgCHQACQaADQCAIdAAIBIEOAIEg0AEgEAQ6AASCQAeAQBDoABAIAh0AAkGgA0AgCHQACASBDgCBINABIBAEOgAEgkAHgEAQ6AAQCAIdAAJBoANAIAh0AAgEgQ4AgSDQASAQBDoABCJRoJvZIjPbamadZraiwPOzzOwxM3vGzDaa2XXplwoAGEjRQDezakkrJS2WNF9Sq5nNz1v2cUlr3P0SSTdK+kLahQIABpZkC/0ySZ3uvt3dT0p6SNLSvDUu6azs40mSdqdXIgAgiSSB3ihpZ864KzuX6xOS3mdmXZLWSvrrQi9kZsvMrMPMOvbv3z+EcgEA/UnroGirpAfcfaak6yR9zcxir+3ube7e4u4tDQ0NKb01AEBKFui7JDXljGdm53LdKmmNJLn7zyTVSpqaRoEAgGSSBPpTkprNbLaZjVXmoGd73poXJL1dkszs9coEOvtUAGAEFQ10dz8l6XZJ6yRtUebbLJvN7B4zW5Jd9g+SPmBmv5S0WtL73d2Hq2gAQFxNkkXuvlaZg525c3flPH5W0hXplgYAGAzOFAWAQBDoABAIAh0AAkGgA0AgCHQACASBDgCBINABIBAEOgAEgkAHgEAQ6AAQCAIdAAJBoANAIAh0AAgEgQ4AgSDQASAQBDoABIJAB4BAEOgAEAgCHQACQaADQCAIdAAIBIEOAIEg0AEgEAQ6AASCQAeAQBDoABAIAh0AAkGgA0AgCHQACASBPvzOMrOtZtZpZitKXUw5MLNF9CSKnhRGXwanptQFhKy7u1uSZkmaL6lL0lNm1u7uzxZab2Y+guWVUo+kZiXoiVRRfblA9CRf4s9KBfXkgLs3FHqCQB9GTz75pCSdcPftkmRmD0laKqnfX1SpeiRKKyGX1HN4cD2Rwu6LS+oRPck3lM9K6D2RpO7f9PcMu1yG0a5duyTpZM5Ul6TG3DVmtszMOsysYyRrK7EBeyJVbF960ZM+/P4MAlvoJebubZLapIr6J2NR9CWOnsTRk6hEW+hJDkyY2Q1m9qyZbTazr6db5ujU2NgoSWNzpmZK2lWaasoKPRkYPenDZ2UQiga6mVVLWilpsTIH91rNbH7emmZJd0i6wt0XSPq79EsdfS699FJJqjWz2WY2VtKNktpLW1VZoCcF0JOC+KwMQpJdLpdJ6ixyYOIDkla6+0uS5O770i50NKqpqZGkFyStU+ZozSp331zSokrOJHqSx3of0JMIPiuDlSTQGyXtzBl3SXpL3pq5kmRmP1Gm8Z9w9+/lv5CZLZO0TJJmzZo1lHpHo1fcvaXURZQZelKAu88tdQ1liM/KIKT1LZcaZb4r+jZJrZLuN7Oz8xe5e5u7t7h7S0NDwa9RAgCGKEmg75LUlDMudGCiS1K7u7/m7jskPadMwANAP6zAD85EkkB/SlJzkQMT31Fm61xmNlWZXTDb0ysTAFBM0UB391OSblfmwMQWSWvcfbOZ3WNmS7LL1kk6aGbPSnpM0kfc/eBwFQ0AiEt0YpG7r5W0Nm/urpzHLunD2R8AQAlwpigkSZ+a/ZexuTt3fGnQr/ONN7w3NvenG/9zSDWhPL1r4l/F5r7z6hcH/TofnPah2Nx9+1YOqaZS+96l18fm3rs5utf54NFnir7O8X+eEJubdndTZPzb4/1fyoZruQBAIAh0AAgEgQ4AgSDQASAQHBSFJGn5tt+Nzd1ZFT0oeuyTE2Nrujoi12nT676xMP7itaP3oOht02+LzV117qHI+JGdk2NrTnZHr+S6uPFIbM2/7+i7GcMvjn97qCUOu7vP+2BkvGH/qdiaw3dOiYxbP/8nsTWPHr4vMi50AHRO3eLIePuR7yaucyR9rvkvIuMpdS/E1iw/99LI+O+/FP8md8/T+yPjmuXxg8sr2n4aGd+5g4OiABA8Ah0AAkGgA0Ag2IdeoU5+MXoz3fdMeqLonxn/j68WmP15dFj78wJrRo+6cRdExnPqX4utWdj8XGR8zVV7YmvG1B+LjA/vjl9ddNurV/c93lMeF6Zqqr86Nvfisehn5aBeia2p/1T+/uH7YmuSqBklkfShFV+JjMfc2l30z3z02gQvvOLm2NTDb2xNWhZb6AAQCgIdAAJBoANAIAh0AAjE6DgCgTOy8R1vjc29dvXYyHgoV8sL0d82vCMyrqs5HlvjHj2A+eqeqbE19Q0vRcbdp6pja66f96vTj7/5cvx9RsLZ4y+OjC+tuii2Jv8EoCkTLhm2eq6ui95r+Ln4+VjDzvJi8dinx8TW3Pfp6FVFJ4z7YWzN0RO/TqWeRwucuNYfttABIBAEOgAEgkAHgECwD32U+b3xt0TGi6fH9+9tPxz9z/r1zR5b8+mLvpBuYaPAggnRu8q885zpsTVTa6MnEl00+UBsjVm0n9U18YtVHf9tXWTcfTL+32nOm/ousjRu40jsQ4+fvPTysU2R8bfyxoUkufPOUD12pCu115pRd0Vs7uON8yLjG976o9ias6+K1rD0AzfF1qw9Mvi7eQ3Vg4eS38WJLXQACASBDgCBINABIBAEOgAEgoOiZeQNU2r1/SV9V/s79yu/iq358bFV0fGvh7uq0htTVa+G8S2nx5MUP5FnbtW5kfGFE+MHAJsmRA949ih+JcWmCdEzWaad/VJszTmNeyPj2rw7GElSVf2J6ERPvB6b0neykU0ofrW+qGpVVfXdQaqnp9CVMPPFD47Xjp0ZGR8/md5ByaHYeuSRIf/ZC8ZP1r/O67vj0QuH62Nr3jR9d2T87v+4Mrbmx/evyptJ5wDo9LrLY3N7jxS/yulgsIUOAIEg0AEgEAQ6AASCfehlZOPB4wX3m1e6pnFjde8FTafHM86K3zHn5KnovtHGhv2xNePGR+8iNOGsw7E1VdU9kfGkd8RPLNL4CZHhieb4XX5q50Tven/kYIF9pVV9v349n/l6/PkBXHLxRG1o73vfMQ+vi61Z8onoCTHfP9oWWzOUfebLGz8Um6vJO0Rwzwtvia+pit+NJ03bjh3Su5956PR4Ym1zbM2rzz8fGRe60NgfTFgWGZ9XFz8p7KsvfzsyHltdF1szvXpuZDzOa2Nr5oyP9uR1Y+Kv883fJj8JkC10AAgEgQ4AgSDQASAQBHoKbrnlFk2bNk0XX9x3s4BDhw7p2muvlaSLzewHZnZOyQosiR5J3dmfXt47rsie3PY363XB67+sy3+/b3/5oZeOa+kff0vPbjmoSuxJBp+VtJh7/GSDkdDS0uIdHR0lee+0bdiwQfX19brpppu0aVPmanXLly/X5MmTdccddzwt6WFJ57j7Rwd6HTNzKX5nm3LxR/UfjM09evi+flb3fq561Pd36j3g6Il7IsX7UuhA1mxfEBlXF9hWOWHRE4max8QzYlpt9OjenmPx348dr70cGV8xKf4672zaExkffW2MNh/eq9qqMfr8Cz/V5+b9oX5x6Bz94KWfaHxVrbYc69SLJ/ffoSH2ZNXr3x9b82c3rYn+mfHxE6m6NrwpMv7RlgWxNdddHj2gO+WqbbE1T95/bWT86vHxsTWTxh+NjOdcuF2S9LM9R1RXU6XbN+zWM/+XOfnmY5/dpnPOGqO7PrcjmN+f9HQ/7e4thZ5hCz0FV155pSZPjt4m6pFHHtHNN58+gv2gpHeNcFklFj8zMhPyp+crricL6qdrYnX01n/PHduhN9afvqRrxfVEkn7n3DqdPS4axN997KDeu/T05Y0rsi9DwdcWh8nevXs1Y8aM3uEeSfGLb0sys2WSlhV6LkynA73fnkiV05fD3Uc1se8rb/Qka9/Bk5rRMK53yO9PQom20M1skZltNbNOM1sxwLrrzczNrOA/ByqVZ/ZrFdy35e5t7t7S3z+hQjVQT7LPV1xf6Elh/P4MgrsP+KPMTqltkuZIGivpl5LmF1g3UdIGSU9Iain2ugsXLvSQ7NixwxcsWHB6PHfuXN+9e7dL6pA0Q9JWL95rl6oD+qny6N9JvXOJexJeX/rtiVduT9L5rITXk/5+1NFfD5JsoV8mqdPdt7v7SUkPSVpaYN0nJd0raSTupVX2lixZogcffLB3eLOkoV9GLhimnA0teiKJnvSHvgxFkn3ojZJ25oy7JEXO6zWzN0tqcvf/NrOP9PdCufu7Zs2aNfhqy1Rra6sef/xxHThwQDNnztTdd9+tFStW6IYbbpCkiyW9LOmGkhY54nrU9wvZrcwvqGXn6UmBnkjSNaq4nkh8VtJzxgdFzaxK0mclvb/YWndvk9QmZb62eKbvXS5Wr15dcH79+vUys03ufs0Il1QG+vvHX7WkbnoSUS2pW5XZE4nPSnqS7HLZJakpZzwzO9drojL/L/q4mf1a0uWS2jkwCgAjK0mgPyWp2cxmm9lYSTdKau990t1fcfep7n6+u5+vzEHRJe4exllDADBKFA10dz8l6XZJ6yRtkbTG3Teb2T1mtmS4CwQAJJNoH7q7r5W0Nm/urn7Wvu3MywIADBan/gNAIDj1v7wckLp/k308NTMeNQZT73mDfO3evtCTPqO1J9Lw9aXie0KglxF3b+h9bGYdPopOZx7Oenv7Qk/6jNaeSMNXMz1hlwsABINAB4BAEOjlK36L9vI2EvXSk9K8R9qGu+aK7Ql3LBpmZtbv3UUAIE1soQNAIAh0AAgEgV5mkt4dqpTMbJWZ7TOzTTlzk7N3Z38+7bu005N+37Os+0JPChvOvhDoZcTMqiWtlLRY0nxJrWY2v7RVFfSApEV5cyskrXf3Zknrs+MzRk8KGyV9eUD0pJAHNEx9IdDLS9K7Q5WUu2+QdChveqkyd2eX0r1LOz0prOz7Qk8KG86+EOjlpdDdoRpLVMtgTXf3F7OPB7x7/SDRk8JGa1/oSWGp9IVAR+rcB757fSWiJ3H0pLAz6QuBXl6K3R2qnO01sxmSlP3ffSm9Lj0pbLT2hZ4UlkpfCPTyMuDdocpcuzJ3Z5fSvUs7PSlstPaFnhSWTl/cvSQ/Cxcu9EogqcMH0RdJ10l6TtI2SR8bzJ8dqR9JqyW9KOk1ZfZT3ippijJH55+X9ENJk1N8P3oyCvtCT0a+L5z6P8w49R/ASGGXCwAEgkAHgEAQ6AAQCAIdAAJBoANAIAh0AAgEgQ4AgSDQASAQBDoABIJAB4BAEOgAEAgCHQACQaADQCAIdAAIBIEOAIEg0AEgEIkC3cwWmdlWM+s0sxUFnv+wmT1rZhvNbL2ZnZd+qQCAgRQNdDOrlrRS0mJJ8yW1mtn8vGXPSGpx9zdIeljSv6RdKABgYEm20C+T1Onu2939pKSHJC3NXeDuj7n70ezwCWXutg0AGEFJAr1R0s6ccVd2rj+3SvqfQk+Y2TIz6zCzjv379yevEgBQVKoHRc3sfZJaJH2m0PPu3ubuLe7e0tDQkOZbA0DFq0mwZpekppzxzOxchJldI+ljkt7q7ifSKQ8AkFSSLfSnJDWb2WwzGyvpRkntuQvM7BJJX5K0xN33pV8mAKCYooHu7qck3S5pnaQtkta4+2Yzu8fMlmSXfUZSvaT/MrNfmFl7Py8HABgmSXa5yN3XSlqbN3dXzuNrUq4LADBInCkKAIEg0AEgEAQ6AASCQAeAQBDoABAIAh0AAkGgA0AgCHQACASBDgCBINABIBAEOgAEgkAHgEAQ6AAQCAIdAAJBoANAIAh0AAgEgQ4AgSDQASAQBDoABIJAB4BAEOgAEAgCHQACQaADQCAIdAAIBIEOAIEg0AEgEAQ6AASCQAeAQBDoABAIAh0AAkGgA0AgCHQACASBDgCBINABIBAEOgAEIlGgm9kiM9tqZp1mtqLA8+PM7BvZ539uZuenXikAYEBFA93MqiWtlLRY0nxJrWY2P2/ZrZJecvcLJf2bpHvTLhQAMLAkW+iXSep09+3uflLSQ5KW5q1ZKunB7OOHJb3dzCy9MgEAxdQkWNMoaWfOuEvSW/pb4+6nzOwVSVMkHchdZGbLJC3LDk+Y2aahFD3KXFTqAgBUhiSBnhp3b5PUJklm1uHuLSP5/qVgZh2lrgFAZUiyy2WXpKac8czsXME1ZlYjaZKkg2kUCABIJkmgPyWp2cxmm9lYSTdKas9b0y7p5uzj90j6X3f39MoEABRTdJdLdp/47ZLWSaqWtMrdN5vZPZI63L1d0pclfc3MOiUdUib0i2k7g7pHk0r5ewIoMWNDGgDCwJmiABAIAh0AAlGSQC92KYEQmNkqM9tXId+1B1AGRjzQE15KIAQPSFpU6iIAVI5SbKEnuZTAqOfuG5T5xg8AjIhSBHqhSwk0lqAOAAgKB0UBIBClCPQklxIAAAxSKQI9yaUEAACDNOKB7u6nJPVeSmCLpDXuvnmk6xhuZrZa0s8kXWRmXWZ2a6lrAhA2Tv0HgEBwUBQAAkGgA0AgCHQACASBDgCBINABIBAEOgAEgkAHgED8P54HqaB1suoIAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "f2, axarr2 = plt.subplots(1,5)\n",
    "for i in range(1,5):\n",
    "    # plt.imshow(pred[1][0,:,:,1]) \n",
    "    # f1 = activation_model.predict(test_images[0].reshape(1, 28, 28, 1))[1] # 选取第一层\n",
    "    f4 = pred[1]\n",
    "    img = f4[0, : , :, i]  # 不同的过滤器 第一个是靴子的斜面，第二个是靴子，第三个靴子底部\n",
    "    # print(img.shape)\n",
    "    axarr2[i].imshow(img, cmap='inferno')\n",
    "    axarr2[i].grid(False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x28144efb4c8>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAMSUlEQVR4nO3df6jd9X3H8eerSUxM6tS0m6tGZtjEIrKa7q5Vu+loHLNWTKH7Q5nDdsL9Z1vTUigRB2WwwWBFKqxUgj9ZRf+wbhVxrS79tTGXedXQqbE1s06jSZMiU3HUJPjeH+cIt3eJSc/3e37o5/mAyznne875fl7c3Fe+3/M953y+qSokvfO9a9oBJE2GZZcaYdmlRlh2qRGWXWrE8kkOdlxW1irWTHJIqSk/4zUO1Os53H0TLfsq1vDhbJzkkFJTtte2I97nbrzUCMsuNcKyS42w7FIjOpU9ySVJfphkV5ItfYWS1L+Ry55kGfAV4GPA2cCVSc7uK5ikfnXZsn8I2FVVz1TVAeAuYFM/sST1rUvZTwOeX3R793DZz0kyn2QhycJBXu8wnKQuxn6Arqq2VtVcVc2tYOW4h5N0BF3K/gJw+qLb64bLJM2gLmV/GDgzyfokxwFXAPf2E0tS30b+bHxVHUryZ8C3gGXALVX1RG/JJPWq0xdhqup+4P6eskgaIz9BJzXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71Igu52c/Pcl3kjyZ5Ikkm/sMJqlfXc4Icwj4fFU9muQE4JEkD1bVkz1lk9SjkbfsVbWnqh4dXn8V2Mlhzs8uaTZ0Otfbm5KcAWwAth/mvnlgHmAVq/sYTtIIOh+gS/Ju4OvAZ6vqlaX3V9XWqpqrqrkVrOw6nKQRdSp7khUMin5HVd3TTyRJ49DlaHyAm4GdVXV9f5EkjUOXLftHgD8GPppkx/Dn0p5ySerZyAfoqupfgfSYRdIY+Qk6qRGWXWpEL++zq117N1/QeR2/esO/9ZBk+p79q/M7r+OMv3iohySH55ZdaoRllxph2aVGWHapEZZdaoRllxph2aVGWHapEZZdaoRllxph2aVGWHapEZZdaoRllxph2aVGWHapEamqiQ32S1lbH87GiY2n8fvWizs6r+MPTj238zpO+Jf3dl7HjufXdXv+hTd2zvDJded1ev722sYr9dJh54Z0yy41wrJLjbDsUiMsu9SIPk7suCzJY0nu6yOQpPHoY8u+mcG52SXNsK5ncV0HfBy4qZ84ksal65b9y8AXgDe6R5E0Tl1O2XwZsK+qHjnK4+aTLCRZOMjrow4nqaOup2y+PMmzwF0MTt38taUPqqqtVTVXVXMrWNlhOEldjFz2qrq2qtZV1RnAFcC3q+qq3pJJ6pXvs0uN6OUsrlX1XeC7faxL0ni4ZZcaYdmlRlh2qRG9vGbX29eZD3d7O/Si+fnOGVbxH53X8erv/rTzOn6dbuv4JN0mnhg3t+xSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNsOxSIyy71AjLLjXCskuNcPKKt7Ef3fpbnddx3Xtu7vT8p+/zxB9vF27ZpUZYdqkRll1qhGWXGtH1/OwnJbk7yVNJdiY5v69gkvrV9Wj8DcA3q+oPkxwHrO4hk6QxGLnsSU4ELgQ+BVBVB4AD/cSS1Lcuu/Hrgf3ArUkeS3JTkjVLH5RkPslCkoWD+J6sNC1dyr4c+CDw1araALwGbFn6oKraWlVzVTW3gm5nH5E0ui5l3w3srqrtw9t3Myi/pBk0ctmrai/wfJKzhos2Ak/2kkpS77oejf9z4I7hkfhngE93jyRpHDqVvap2AHP9RJE0Tn6CTmqEZZca0dz32d+4aEPndez/wPGd13H8/jc6r2PtQ+m8jr/+9Lmd16G3B7fsUiMsu9QIyy41wrJLjbDsUiMsu9QIyy41wrJLjbDsUiMsu9QIyy41wrJLjbDsUiMsu9QIyy41wrJLjZjo5BWHfmMV+65/f6d1/Mqmpzo9/13fe6zT8wFO+V7nVUgT55ZdaoRllxph2aVGWHapEZ3KnuRzSZ5I8niSO5Os6iuYpH6NXPYkpwGfAeaq6hxgGXBFX8Ek9avrbvxy4Pgky4HVwIvdI0kahy5ncX0B+BLwHLAHeLmqHugrmKR+ddmNPxnYBKwHTgXWJLnqMI+bT7KQZOHQy6+NnlRSJ1124y8GflxV+6vqIHAPcMHSB1XV1qqaq6q55Seu6TCcpC66lP054Lwkq5ME2Ajs7CeWpL51ec2+HbgbeBT4z+G6tvaUS1LPOn0Rpqq+CHyxpyySxshP0EmNsOxSIyy71IiJTl6xfNfPOk8+IWk0btmlRlh2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaMdHJK6Rxef3S3+68jpX3P9xDktnlll1qhGWXGmHZpUZYdqkRRy17kluS7Evy+KJla5M8mOTp4eXJ440pqatj2bLfBlyyZNkWYFtVnQlsG96WNMOOWvaq+j7w0pLFm4Dbh9dvBz7RbyxJfRv1ffZTqmrP8Ppe4JQjPTDJPDAPsIrVIw4nqavOB+iqqoB6i/u3VtVcVc2tYGXX4SSNaNSy/yTJ+wCGl/v6iyRpHEYt+73A1cPrVwPf6CeOpHE5lrfe7gQeAs5KsjvJNcDfAL+f5Gng4uFtSTPsqAfoqurKI9y1secsksbIT9BJjbDsUiP8PrveEd7p30Xvg1t2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaYdmlRlh2qRGWXWqEZZcaYdmlRlh2qRHHckaYW5LsS/L4omV/m+SpJD9I8g9JThprSkmdHcuW/TbgkiXLHgTOqarfBH4EXNtzLkk9O2rZq+r7wEtLlj1QVYeGN/8dWDeGbJJ61Mdr9j8B/ulIdyaZT7KQZOEgr/cwnKRRdCp7kuuAQ8AdR3pMVW2tqrmqmlvByi7DSepg5NM/JfkUcBmwsaqqt0SSxmKksie5BPgCcFFV/W+/kSSNw7G89XYn8BBwVpLdSa4B/g44AXgwyY4kN445p6SOjrplr6orD7P45jFkkTRGfoJOaoRllxph2aVGZJLvmiXZD/z3WzzkvcBPJxTnrcxCjlnIALORYxYywGzkOFqGX6uqXz7cHRMt+9EkWaiqOXPMRoZZyTELGWYlR5cM7sZLjbDsUiNmrexbpx1gaBZyzEIGmI0cs5ABZiPHyBlm6jW7pPGZtS27pDGx7FIjZqbsSS5J8sMku5JsmcL4pyf5TpInkzyRZPOkMyzJsyzJY0num9L4JyW5ezjX4M4k508px+eG/x6PJ7kzyaoJjHm4eRfXJnkwydPDy5OnlGPk+R9nouxJlgFfAT4GnA1cmeTsCcc4BHy+qs4GzgP+dAoZFtsM7Jzi+DcA36yq9wMfmEaWJKcBnwHmquocYBlwxQSGvo3/P+/iFmBbVZ0JbBvenkaOked/nImyAx8CdlXVM1V1ALgL2DTJAFW1p6oeHV5/lcEf92mTzPCmJOuAjwM3TWn8E4ELGX67saoOVNX/TCMLg29mHp9kObAaeHHcAx5u3kUGf4+3D6/fDnxiGjm6zP84K2U/DXh+0e3dTKloAEnOADYA26cU4csMJgd5Y0rjrwf2A7cOX0rclGTNpENU1QvAl4DngD3Ay1X1wKRzDJ1SVXuG1/cCp0wpx2JvOf/jUrNS9pmR5N3A14HPVtUrUxj/MmBfVT0y6bEXWQ58EPhqVW0AXmMyu60/Z/i6eBOD/3xOBdYkuWrSOZYaTsM21fesj2X+x6VmpewvAKcvur1uuGyikqxgUPQ7quqeSY8/9BHg8iTPMng589EkX5twht3A7qp6c8/mbgbln7SLgR9X1f6qOgjcA1wwhRwAP0nyPoDh5b4p5Vg8/+Mf/SLzP85K2R8GzkyyPslxDA7C3DvJAEnC4DXqzqq6fpJjL1ZV11bVuqo6g8Hv4dtVNdGtWVXtBZ5PctZw0UbgyUlmGHoOOC/J6uG/z0amd9DyXuDq4fWrgW9MI8Si+R8v/4Xnf6yqmfgBLmVwdPG/gOumMP7vMNg1+wGwY/hz6ZR/J78H3Delsc8FFoa/j38ETp5Sjr8EngIeB/4eWDmBMe9kcIzgIIO9nGuA9zA4Cv808M/A2inl2MXg+Nabf6M3Huv6/Lis1IhZ2Y2XNGaWXWqEZZcaYdmlRlh2qRGWXWqEZZca8X8UJ2iSoBn9KwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "f4 = pred[1]\n",
    "img = f4[0, : , :, i]\n",
    "plt.imshow(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8KVPZqgHo5Ux"
   },
   "source": [
    "EXERCISES\n",
    "\n",
    "1. Try editing the convolutions. Change the 32s to either 16 or 64. What impact will this have on accuracy and/or training time.\n",
    "\n",
    "2. Remove the final Convolution. What impact will this have on accuracy or training time?\n",
    "\n",
    "3. How about adding more Convolutions? What impact do you think this will have? Experiment with it.\n",
    "\n",
    "4. Remove all Convolutions but the first. What impact do you think this will have? Experiment with it. \n",
    "\n",
    "5. In the previous lesson you implemented a callback to check on the loss function and to cancel training once it hit a certain amount. See if you can implement that here!\n",
    "\n",
    "练习\n",
    "\n",
    "1. 尝试修改卷积层参数。将32改为16或64。这对准确度和/或训练时间有什么影响？\n",
    "\n",
    "2. 删除最后的卷积层。这将对精度或训练时间产生什么影响？\n",
    "\n",
    "3. 增加更多的卷积层会有什么影响？实验一下吧。\n",
    "\n",
    "4. 除第一项外，删除所有的Convolutions。这样做会有什么影响？请完成实验。\n",
    "\n",
    "5. 在上一节课中，实现了通过一个回调函数来检查模型的损失，并在损失减小到一定量时取消训练。看看是否能在这里实现？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.1.0\n",
      "(60000, 28, 28)\n",
      "(10000, 28, 28)\n",
      "Model: \"sequential_8\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_12 (Conv2D)           (None, 26, 26, 32)        320       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_10 (MaxPooling (None, 13, 13, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_13 (Conv2D)           (None, 11, 11, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_11 (MaxPooling (None, 5, 5, 64)          0         \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 1600)              0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 128)               204928    \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 225,034\n",
      "Trainable params: 225,034\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "# from tensorflow import keras\n",
    "\n",
    "print(tf.__version__)\n",
    "# 选择数据集\n",
    "mnist = tf.keras.datasets.mnist\n",
    "# 切分数据集\n",
    "(training_images, training_labels), (test_images,test_labels) = mnist.load_data()\n",
    "\n",
    "print(training_images.shape)\n",
    "# 这是因为第一次卷积期望一个包含所有数据的单一张量，所以要把训练数据设置为60000x28x28x1的一个4D列表，\n",
    "# 测试图像也是如此处理。如果不这样做，会在训练时得到一个错误，因为卷积操作将不能识别数据形状。\n",
    "training_images = training_images.reshape(60000, 28, 28,1)\n",
    "# 训练集进行归一化处理\n",
    "training_images = training_images/255.0\n",
    "\n",
    "print(test_images.shape)\n",
    "test_images = test_images.reshape(10000, 28, 28,1)\n",
    "# 测试集进行归一化处理\n",
    "test_images = test_images/255.0\n",
    "# 定义模型\n",
    "model = tf.keras.models.Sequential()\n",
    "# 1、添加一个两维的卷积层 过滤器的数量为32，过滤器的大小为3*3 输入的数据的形状为28*28*1,\n",
    "# 卷积之后每个图片的像素会小2即26*26，以为过滤器的大小为3*3 \n",
    "# Param (3*3+1) * 32 = 320\n",
    "model.add(tf.keras.layers.Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))\n",
    "# 2、加上一个maxpool层（池化层），用来压缩图像，同事保持卷所强调的特征内容，通过指定为（2,2），将图像的大小缩小为四分之一。\n",
    "# 它的想法是创建一个2x2的像素数组，然后选取最大的一个，从而将4个像素变成1个，在整个图像中重复这样做，这样做的结果是将水平像素的数量减半，垂直像素的数量减半，有效地将图像缩小25%。\n",
    "# 即26的一半/13 \n",
    "\n",
    "\n",
    "model.add(tf.keras.layers.MaxPool2D(2,2))\n",
    "# 3和4层，在来一次\n",
    "# 第3层 调整参数Param (3*3*32+1)*64 = 18496个过滤器=\n",
    "model.add(tf.keras.layers.Conv2D(64,(3,3),activation='relu'))\n",
    "model.add(tf.keras.layers.MaxPool2D(2,2))\n",
    "\n",
    "# 5层对输出进行扁平化处理\n",
    "model.add(tf.keras.layers.Flatten())\n",
    "# 5层的输出为5*5*64 = 1600\n",
    "\n",
    "# 6层增加一个128个神经元的全连接层，\n",
    "model.add(tf.keras.layers.Dense(units=128,activation='relu'))\n",
    "# 该层的输出 (1600+1)*128 = 204928\n",
    "\n",
    "\n",
    "# 7层增加一个10个神经元的输出层\n",
    "model.add(tf.keras.layers.Dense(10,activation='softmax'))\n",
    "# 输出为10个类型\n",
    "model.summary()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 21s 358us/sample - loss: 0.1240 - accuracy: 0.9626\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 18s 306us/sample - loss: 0.0410 - accuracy: 0.9873\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 18s 301us/sample - loss: 0.0275 - accuracy: 0.9913\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 18s 302us/sample - loss: 0.0187 - accuracy: 0.9938\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 18s 303us/sample - loss: 0.0146 - accuracy: 0.9955\n",
      "10000/10000 [==============================] - 2s 176us/sample - loss: 0.0310 - accuracy: 0.9909\n",
      "0.9909\n"
     ]
    }
   ],
   "source": [
    "# 现在开始编译模型，调用model.fit方法做训练，接着用测试集评估损失和准确率\n",
    "model.compile(optimizer='adam', loss=tf.keras.losses.sparse_categorical_crossentropy,metrics=['accuracy'])\n",
    "model.fit(training_images,training_labels,epochs=5)\n",
    "test_loss, test_acc=model.evaluate(test_images,test_labels)\n",
    "print(test_acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "23b34fee034e6c6559cf1b732e166f57ae608166b2226a83e90a4ecbc0876e39"
  },
  "kernelspec": {
   "display_name": "Python 3.7.13 ('learn_tensorflow')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
